NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY,  CALIFORNIA 

THESIS 


IDENTIFICATION  OF  HUMAN  FACTORS  CONCERNS  IN 

JOINT  STRIKE  FIGHTER 

AND  TRAINING  RECOMMENDATIONS 

by 

Arif  Batu  Bitik 
Bulent  Kulunk 

September  2008 

Thesis  Advisor: 

Joseph  Sullivan 

Co-Advisor: 

Michael  McCauley 

Second  Reader: 

Anthony  Ciavarelli 

This  thesis  was  done  at  the  MOVES  Institute 
Approved  for  public  release;  distribution  is  unlimited 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


Form  Approved  OMB  No. 

0704-0188 _ 

Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing 
instruction,  searching  existing  data  sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection 
of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including 
suggestions  for  reducing  this  burden,  to  Washington  headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215 
Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction 
Project  (0704-0188)  Washington  DC  20503. 

I .  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

_ September  2008 _ Master’s  Thesis _ 

4.  TITLE  AND  SUBTITLE  Identification  of  Human  Factors  Concerns  in  Joint  5.  FUNDING  NUMBERS 
Strike  Fighter  and  Training  Recommendations _ 

6.  AUTHOR(S)  Arif  Batu  Bitik,  Bulent  Kuiunk  _ 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES)  8.  PERFORMING  ORGANIZATION 

Naval  Postgraduate  School  REPORT  NUMBER 

Monterey,  CA  93943-5000 

9.  SPONSORING  /MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES)  1 0.  SPONSORING/MONITORING 
N/A  AGENCY  REPORT  NUMBER 

II.  SUPPLEMENTARY  NOTES  The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the 

official  policy  or  position  of  the  Department  of  Defense  or  the  U.S.  Government. _ 

12a.  DISTRIBUTION  /  AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE  ” 

Approved  for  public  release;  distribution  is  unlimited _ 

13.  ABSTRACT  (maximum  200  words) 

Military  aviation  is  the  frontier  of  implementing  leading  edge  technology.  The  major  objectives  of 
advanced  technology  aircraft  are  to  increase  pilot  safety  and  mission  efficiency;  the  Joint  Strike  Fighter,  the 
most  modern  fighter  aircraft  under  development,  has  many  technological  innovations  for  just  this  purpose. 

A  common  fact  is  that  technology  develops  and  is  used  faster  than  it  can  be  researched 
thoroughly.  This  thesis  seeks  to  identify  and  mitigate  potential  human  factors  concerns  related  to  the  Joint 
Strike  Fighter,  before  it  is  used  in  the  air  forces  of  participating  countries.  The  objective  is  neither  to  blame 
nor  defend  the  design  of  the  aircraft. 

Two  surveys  and  an  interview  yielded  the  following  findings:  fighter  pilots  will  use  automation  more 
in  JSF  than  in  their  current  types,  the  main  LCD  management  will  be  key  to  mission  efficiency  and  safety, 
the  Distributed  Aperture  System  should  be  addressed  very  carefully  to  avoid  disorientation  issues,  and 
tactical  decision-making  skills  will  be  more  important  and  demanding. 

New  approaches  for  better  automation  training,  more  focus  on  data  filtering,  display  management, 
prioritization  skills,  establishing  robust  standard  operating  procedures  for  DAS,  and  addressing  the 
complex  decision-making  skills  in  more  detail  than  the  current  training  curriculums  are  concluded  to  be  the 
major  requirements  of  JSF  pilot  training. 

14.  SUBJECT  TERMS  Human  Factors,  Cockpit  Automation,  Pilot  Workload,  Attentional  15.  NUMBER  OF 

Resources,  Joint  Strike  Fighter,  DAS  PAGES 

161 

16.  PRICE  CODE 

17.  SECURITY  18.  SECURITY  19.  SECURITY  20.  LIMITATION  OF 

CLASSIFICATION  OF  CLASSIFICATION  OF  THIS  CLASSIFICATION  OF  ABSTRACT 

REPORT  PAGE  ABSTRACT 

_ Unclassified _ Unclassified _ Unclassified _ LJU _ 

NSN  7540-01-280-5500  Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


I 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Approved  for  public  release;  distribution  is  unlimited 


IDENTIFICATION  OF  HUMAN  FACTORS  CONCERNS  IN  JOINT  STRIKE 
FIGHTER  AND  TRAINING  RECOMMENDATIONS 

Arif  B.  Bitik 

Captain,  Turkish  Air  Force 
B.S.,  Turkish  Air  Force  Academy,  1999 

Bulent  Kulunk 

First  Lieutenant,  Turkish  Air  Force 
B.S.,  Turkish  Air  Force  Academy,  2001 

Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 


MASTER  OF  SCIENCE  IN 

MODELING,  VIRTUAL  ENVIRONMENTS,  AND  SIMULATION  (MOVES) 


from  the 


NAVAL  POSTGRADUATE  SCHOOL 
September  2008 


Author:  Arif  Batu  Bitik 

Bulent  Kulunk 

Approved  by:  Joseph  Sullivan 

Thesis  Advisor 

Michael  McCauley 
Co-Advisor 

Anthony  Ciavarelli 
Second  Reader 

Mathias  Kolsch 

Chair,  MOVES  Academic  Committee 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


IV 


ABSTRACT 


Military  aviation  is  the  frontier  of  implementing  leading  edge  technology. 
The  major  objectives  of  advanced  technology  aircraft  are  to  increase  pilot  safety 
and  mission  efficiency;  the  Joint  Strike  Fighter,  the  most  modern  fighter  aircraft 
under  development,  has  many  technological  innovations  for  just  this  purpose. 

A  common  fact  is  that  technology  develops  and  is  used  faster  than  it  can 
be  researched  thoroughly.  This  thesis  seeks  to  identify  and  mitigate  potential 
human  factors  concerns  related  to  the  Joint  Strike  Fighter,  before  it  is  used  in  the 
air  forces  of  participating  countries.  The  objective  is  neither  to  blame  nor  defend 
the  design  of  the  aircraft. 

Two  surveys  and  an  interview  yielded  the  following  findings:  fighter  pilots 
will  use  automation  more  in  JSF  than  in  their  current  types,  the  main  LCD 
management  will  be  key  to  mission  efficiency  and  safety,  the  Distributed  Aperture 
System  should  be  addressed  very  carefully  to  avoid  disorientation  issues,  and 
tactical  decision-making  skills  will  be  more  important  and  demanding. 

New  approaches  for  better  automation  training,  more  focus  on  data 
filtering,  display  management,  prioritization  skills,  establishing  robust  standard 
operating  procedures  for  DAS,  and  addressing  the  complex  decision-making 
skills  in  more  detail  than  the  current  training  curriculums  are  concluded  to  be  the 
major  requirements  of  JSF  pilot  training. 
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EXECUTIVE  SUMMARY 


Aviation  is  the  frontier  of  implementing  leading  edge  technology.  Modern 
systems,  displays  and  interfaces  continuously  enter  cockpits  to  increase  mission 
capabilities  as  well  as  safety  and  efficiency.  However,  it  is  up  to  the  pilots  to  meet 
the  promised  objectives  using  the  new  systems. 

The  military  requirements  and  budget  concerns  during  the  design  and 
acquisition  phases  generally  limit  the  conduct  of  relative  research  on  human 
factors  issues  about  the  new  systems;  thus,  many  issues  are  still  being 
addressed  long  after  the  acceptance  of  the  assets  or  their  subsystems.  In  some 
cases  the  drawbacks  result  in  mishaps,  and  in  others  less  efficiency  than 
expected,  until  the  human  factors  problems  are  solved  completely. 

Military  aviation  has  the  most  demanding  requirements  among  the  aviation 
communities.  The  nature  of  the  missions,  the  environment,  and  all  additional 
stressors  are  the  reasons  for  this  fact;  added  to  those  factors  is  the  need  of 
military  pilots  to  build  more  advanced  skills  than  their  civilian  counterparts.  The 
major  task  in  commercial  aviation  is  navigation,  whereas  navigation  is  only  a  tool 
to  reach  and  return  from  mission  areas  in  military  aviation.  Additionally,  military 
pilots  also  accomplish  air  combat  and  night  assaults  out  of  many  other  military 
mission  types.  In  order  to  succeed  in  their  tasks,  the  military  pilots  use  various 
enhanced  systems  that  the  commercial  pilots  don’t. 

Currently  the  Joint  Strike  Fighter  is  the  most  advanced  fighter  aircraft 
under  development.  Many  allied  countries  contribute  to  the  JSF  project,  and  it  is 
predicted  that  it  will  be  the  main  asset  of  the  NATO  air  forces  in  the  coming 
decades.  JSF  has  many  innovations  that  will  have  impact  on  pilot  performance. 
The  increased  use  of  automation  (causing  a  role  shift  for  the  pilot),  a  big  LCD 
display  suite  as  the  main  PVI,  a  clear  cockpit  design  with  far  fewer  controls  and 
switches,  enhanced  systems  such  as  Distributed  Aperture  System  (which  allows 
pilots  to  “see  through”  the  aircraft  fuselage),  and  data  fusion  from  many  sensors 
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are  among  the  main  characteristics  of  the  JSF  cockpit.  All  of  the  aforementioned 
features  are  expected  to  cause  some  human  factors  concerns  that  must  be 
researched  and  analyzed  for  safer  and  more  efficient  utilization. 

The  literature  includes  extensive  research  on  automation,  workload, 
attentional  management,  prioritization  and  the  effects  of  many  systems  such  as 
HUD  and  HMD  on  human  performance.  The  objective  of  this  thesis  is  twofold:  to 
investigate  whether  there  are  areas  prone  to  those  same  problems  found  in  the 
literature,  and  to  anticipate  the  problems  unique  to  JSF  itself.  Other  than  trying  to 
identify  potential  problems,  the  second  objective  is  to  propose  solutions  in  the 
training  phase  that  will  mitigate  the  predicted  issues. 

The  initial  strategy  of  these  authors  was  to  investigate  the  potential 
problems  regarding  the  transition  phase.  The  JSF  pilots  will  be  chosen  among 
the  current  types,  and  the  main  predicted  concerns  evaluated  as  they  relate  to 
the  potential  Negative  Transfer  of  Training  (NTT)  issues  during  transition  phase. 
The  experiences  of  the  pilots  in  their  current  aircraft  types  would  make  them 
prone  to  errors  in  JSF,  and  this  possibility  may  result  in  mishaps  or  decreased 
efficiency.  It  was  deemed  possible  and  a  major  concern  that  the  locations  of  the 
JSF  cockpit  controls  and  their  operating  procedures  would  conflict  with  the  pilots’ 
prior  experience. 

In  order  to  identify  the  crucial  areas  of  potential  NTT  and  initiate  the 
possible  solution  building  process,  the  authors  traveled  to  the  Lockheed  Martin 
facilities  in  Fort  Worth,  TX.  The  trip  provided  some  opportunities  to  investigate 
the  subject.  These  authors  were  able  to  experience  the  unclassified  simulator, 
conduct  a  preliminary  survey  with  fighter  pilots  who  were  experiencing  various 
missions  in  the  full  mission  simulator,  and  interview  those  pilots  about  their 
opinions.  While  expecting  to  return  with  all  required  information  to  complete  the 
thesis  with  solutions,  the  authors  found  that  the  major  problematic  areas  were 
quite  probably  different  than  those  predicted  as  NTT  during  transition  period.  The 
participants  and  the  authors  mostly  agreed  that  NTT  will  not  be  an  important 
issue,  but  that  role  change,  new  systems  and  Pilot  Vehicle  Interface  will  be  the 


crucial  components  about  the  potential  concerns.  Thus  these  authors  decided  to 
investigate  the  problem  identification  process  deeper;  otherwise  the  solutions 
would  not  address  the  actual  issues. 

A  follow-on  survey  was  formed  with  multiple-choice  questions  in  Likert 
scale,  as  well  as  open-ended  questions  to  cover  any  missed  issue  in  multiple- 
choice  questions.  The  objectives  were  to  identify  the  potential  concerns  robustly, 
and  take  opinions  about  the  solutions.  The  following  areas  emerged  as  potential 
concerns:  automation  is  predicted  to  be  used  in  JSF  much  more  than  in  current 
types,  the  Distributed  Aperture  System  (DAS)  is  predicted  to  have  some 
concerns  if  not  addressed  properly  during  SOP  building  and  training  periods,  and 
the  big  LCD  display  suite  management  and  tactical  decision-making  is  predicted 
to  be  crucial  for  mission  safety  and  efficiency.  To  adapt  to  the  role  change 
properly  and  build  the  requisite  cognitive  skills  to  operate  the  enhanced  systems 
in  JSF  (with  its  different  PVI)  are  thought  to  be  the  main  challenges  to  training 
and  following  JSF  pilots  throughout  their  careers. 

The  literature  is  full  of  studies  about  problems  in  cockpit  automation,  but 
mostly  in  commercial  aircraft.  The  JSF  is  predicted  to  be  the  first  fighter  aircraft 
with  considerable  automation  use;  thus  the  findings  of  the  literature  can  be 
tailored  to  confront  any  problems  about  the  automation  in  JSF.  Similarly, 
decision-making  skills,  workload,  attentional  resource  management  and 
prioritization  are  well  studied,  and  the  related  findings  from  these  studies  can  be 
also  used  for  JSF  as  required.  The  only  predicted  problem  that  has  not  been 
researched  directly  is  the  use  of  DAS,  because  it  has  never  before  been  installed 
in  cockpits. 

A  better  approach  to  problem  identification  and  building  solutions  could  be 
utilizing  objective  methods  such  as  experiments.  But  the  security  issues  about 
the  JSF  project  excluded  that  option  in  this  thesis,  and  the  literature  and 
background  of  the  participants  as  well  as  authors  were  the  basis  for  the  solution 
recommendations. 
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This  thesis  follows  a  bottom-up  approach  to  generating  training 
recommendations;  the  potential  human  factors  concerns  are  predicted  by  the 
surveys,  training  objectives  were  established  to  mitigate  the  potential  problematic 
concerns,  and  both  findings  of  literature  and  operational  background  were 
utilized  in  order  to  recommend  ways  to  address  the  objectives  during  the  SOP 
building  and  training  periods. 

The  research  literature  review,  subjective  reports  by  the  participants  and 
operational  experiences  of  these  authors  formed  the  basis  for  the 
recommendations  to  pilot  training  in  JSF.  A  revision  of  the  traditional  training 
curriculums  is  recommended  to  address  both  the  potential  automation-related 
problems  and  to  ensure  that  the  pilots  build  relevant  cognitive  skills,  while 
utilizing  the  ground  training  devices  as  much  as  possible. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

Aviation  is  one  of  the  leading  application  areas  of  technology 
development.  Either  it  is  the  source  of  inventions  itself,  or  implements  already 
developed  technologies  faster  than  most  of  the  other  communities.  These 
technological  advances  are  reflected  in  cockpits  mostly  as  increasing 
automation,  new  flight  displays,  and  advanced  avionics.  The  increasing  number 
of  sensors  provides  much  more  information  to  the  pilots  than  ever  before.  Before, 
pilots  were  required  to  handle  the  throttle,  stick,  and  a  few  manual  systems. 
However,  with  the  development  of  technology,  the  systems  become  more 
complex  and  have  more  sensors  with  greater  range.  These  systems  used  in 
modern  cockpits  include  radar,  Radar  Warning  Receiver  (RWR),  jammers, 
weapon  and  flight  management  systems,  etc.  The  implementation  and  upgrade 
of  these  systems  raise  the  issue  of  their  management,  which  resulted  in  the 
evolution  of  the  conventional  throttle  and  stick  into  the  more  complicated  Hands 
On  Throttle  and  Stick  (HOTAS)  system. 

The  dramatic  increase  in  mission  effectiveness  is  undeniable.  However, 
these  improvements  in  systems  were  associated  with  human  factors  issues, 
besides  their  benefits.  The  fundamental  task  of  pilots  is  still  the  same — to  fly 
safely,  to  evaluate  the  displays  and  information  presented  on  them,  and  to  use 
this  information  effectively  to  make  the  best  available  decision.  One  could 
wonder  where  the  human  factors  issues  arise  from  if  the  task  of  the  pilot  is  still,  in 
essence,  the  same.  The  best  way  to  answer  this  question  would  be  to  consider 
the  differences  in  pilot-vehicle  interfaces  (PVI).  Earlier,  there  were  separate 
displays  for  each  particular  source  of  information,  and  the  pilot  switched  his 
attention  between  displays  frequently  to  continuously  track  and  evaluate  his 
situation  in  the  tactical  arena.  On  the  other  hand,  most  of  the  flight  instruments 
used  in  cockpits  were  manual  and  analog.  Recently,  with  the  advance  of  Liquid 
Crystal  Display  (LCD)  technology  and  sensor  technology,  engineers  could 
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produce  more  effective  ways  of  information  presentation,  not  to  mention  the 
ability  to  gather  more  and  various  kinds  of  information  that  were  not  considered 
before. 

The  pilots  have  access  to  much  more  information,  but  filtering  the 
information  to  suit  the  specific  tasks  became  an  issue,  as  is  managing  the 
displays.  Automation  itself  also  caused  some  new  problems  for  pilots.  On  top  of 
all  these  advancements,  5th  generation  fighters,  particularly  the  Joint  Strike 
Fighter  (JSF)  bring  even  more  modern  and  new  systems  to  life.  Increased  sensor 
and  data  fusion  and  the  Distributed  Aperture  System  (DAS)  (which  enables  pilots 
to  “look  through”  the  fuselage  in  a  full  360-degree  arc)  are  among  those  modern 
systems.  It  also  has  more  sophisticated  automation  as  well  as  a  relatively  clear 
cockpit  with  fewer  controls  compared  to  older  aircraft. 

There  is  a  need  to  identify  and  understand  the  Human  Factors  concerns  of 
the  Joint  Strike  Fighter  clearly  and  robustly  before  addressing  them  in  proper 
ways.  Some  of  the  questions  are  “How  will  the  pilots  filter  the  data  and  focus  only 
the  required  ones?”,  “What  kind  of  new  skills  will  be  needed  for  new  generation 
fighters,”  “Which  areas  of  JSF  are  going  to  cause  problems  to  the  pilots,  and  thus 
need  to  be  addressed  carefully  during  training?”,  or  “What  kind  of  concerns  can 
be  predicted  about  pilot-autopilot  interaction  in  JSF?”  Identifying  the  problems 
and  important  concerns  and  addressing  them  properly  will  allow  pilots  to  benefit 
from  the  new  assets  more  safely  and  efficiently. 

B.  MOTIVATION 

When  examining  the  human  factors  in  aviation,  one  can  find  out  that  most 
of  the  regulations  are  based  on  previous  accidents  and  mishaps.  Adaptation  to 
newly  exposed  systems,  technologies  and  tactics  brings  previously  unknown 
issues  that  generally  result  in  safety  problems,  accidents  and  even  loss  of 
aircrew.  The  majority  of  these  problems  in  the  past  can  be  seen  as  negative 
transfer  of  training  in  the  transition  phase,  because  the  systems  operated  with 
more  or  less  the  same  logic  only  with  different  displays  and  controls.  Therefore, 
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previous  experiences  resulted  in  some  safety  problems  due  to  new  cockpit 
controls.  However,  the  issue  now  is  different.  Advanced  technology  has  offered 
new  systems  that  are  dramatically  different  from  their  predecessors,  both  in 
operating  logic  and  pilot  interface.  The  motivation  of  this  thesis  research  is  to 
anticipate  the  possible  human  factors  issues  beforehand,  which  might  help  to 
avoid  possible  accidents  and  mishaps,  as  well  as  increasing  the  effective  use  of 
systems. 

C.  OBJECTIVES 

This  thesis  attempts  to  provide  a  comprehensive  insight  into  the  human 
factors  issues  of  JSF.  The  main  objective  is  to  identify  the  important  and 
problematic  human  factors  concerns  of  JSF.  After  the  identification  of  the 
problems,  it  will  be  possible  to  address  them  and  propose  solutions  to  those 
areas  in  order  to  increase  the  safety  and  effectiveness  during  the  transition 
periods  and  thereafter.  One  way  of  proposing  solutions  is  to  use  the  literature 
and  previous  knowledge,  and  the  other  way  is  to  address  them  with  specific 
experiments,  or  field  studies.  The  last  and  undesirable  way  is  to  learn  them  via 
experience,  and  the  intent  of  this  thesis  is  to  identify  and  address  them  before 
any  hazardous  situations  happen. 

D.  THESIS  ORGANIZATION 

1.  Introduction.  This  chapter  gives  a  general  outline  of  the  work  and 
defines  the  problem  the  authors  are  trying  to  solve. 

2.  Related  Work  and  Background.  This  chapter  discusses  the  primary 
concerns  related  to  pilot  workload,  cockpit  automation  and  attentional 
resources  theory  and  summarizes  literature  reviews  on  these  fields. 
The  capabilities  and  key  features  of  JSF  are  also  discussed  in  this 
chapter. 
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3.  Methodology.  This  thesis  followed  survey  methodology  for  research, 
and  conducted  two  surveys  and  interviews.  This  chapter  explains  the 
background  ideas  and  structure  of  the  preliminary  and  follow-on 
surveys,  as  well  as  the  interviews. 

4.  Results  and  Discussion.  The  results  of  the  interviews  and  surveys  are 
provided  and  discussed  in  this  chapter. 

5.  Conclusion  and  Recommendations.  After  reviewing  the  key  findings  of 
the  surveys  and  interviews,  the  establishment  of  relevant  training 
objectives  and  how  to  address  them  in  pilot  training  topics  are 
discussed  in  this  chapter.  Additionally,  recommendations  for  future 
research  are  also  included. 
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II.  RELATED  WORK  AND  BACKGROUND 


With  evolving  technology  and  growing  experience  in  all  scientific  fields,  the 
literature  expands  continuously.  This  sometimes  creates  new  fields;  at  other 
times  it  enriches  the  content  of  an  existing  one.  The  main  consequence  of  this 
phenomenon  is  the  increase  of  required  expertise  in  almost  every  research, 
project,  or  field  of  technology. 

The  scope  of  this  thesis  is  the  Human  Factors  concerns  in  the  JSF 
cockpit.  As  mentioned  above,  even  this  scope  entails  having  information  about 
many  Human  Factors  subfields.  Many  researchers  have  studied  modern  glass 
cockpits,  and  two  major  topics  have  emerged  from  the  literature  about  the 
Human  Factors  in  such  an  environment:  cockpit  automation,  and  pilot  workload 
along  with  management  of  attentional  resources. 

Cockpit  automation  is  a  well-studied  topic  in  commercial  aviation,  but  is 
relatively  fresh  for  military  aviation,  especially  for  fighter  cockpits.  Today,  almost 
all  commercial  airplanes  have  very  sophisticated  automation  systems,  and  as 
Olson  (2001)  reported,  they  are  able  to  perform  all  tasks  but  takeoff  and  landing 
automatically.  It  is  legitimate  to  say  that  even  takeoff  and  landing  can  be 
performed  with  newer  automation  capabilities.  Automation  has  also  brought 
some  problems  along  with  its  solutions,  and  all  of  those  have  been  robustly 
researched  in  the  literature. 

The  other  accepted  truth  is  the  increasing  workload  for  the  pilots.  Pilots 
must  learn  and  use  many  more  systems  than  before,  and  the  structure  of  the 
systems  is  more  complex.  All  these  facts,  with  the  increasing  demands  of  the 
mission  environment,  result  in  increased  pilot  workload  in  the  cockpit.  Both  the 
designer  and  operational  communities  strive  to  build  suitable  systems  and 
operational  procedures,  but  there  are  many  causes  for  concern.  Sometimes 
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technology  develops  faster  than  humans  can  adapt,  thus  adding  work  rather  than 
subtracting  it.  Workload  is  a  well-known  and  studied  topic  by  the  human  factors 
and  ergonomics  experts. 

With  the  increased  level  of  automation,  and  many  more  sophisticated  and 
revolutionary  systems,  the  JSF  will  also  require  focus  on  these  topics.  For  this 
reason,  the  related  work  and  background  section  of  this  thesis  is  divided  into  two 
subsections,  including  cockpit  automation  and  pilot  workload. 

A.  COCKPIT  AUTOMATION 

The  role  shift  of  the  pilots  was  one  of  the  objectives  of  the  Lockheed 
Martin  team;  Kent  (2006)  quoted  Skaff’s  statement  that,  “The  F-35  cockpit  design 
is  driven  by  the  desire  to  return  the  pilot  to  the  role  of  tactician”  (para.  12).  This 
goal  alone  makes  it  worthwhile  to  study  findings  about  automation  for  developing 
JSF  training.  This  chapter  reviews  the  findings  of  research  literature  related  to 
cockpit  automation. 

1.  Introduction  to  Cockpit  Automation 

Before  investigating  the  human  factors  concerns,  safety  and  effectiveness 
aspects  of  the  cockpit  automation  thoroughly,  one  has  to  understand  the  reasons 
for  integrating  automation  with  the  cockpits. 

Sarter,  Woods  and  Billings  (1997)  listed  improved  economical  efficiency 
and  precision  as  two  of  the  general  benefits  of  automation.  However,  they  also 
add  that  these  benefits  have  introduced  other  problems,  and  the  importance  of 
the  human  user  has  been  reinforced,  even  in  the  highly  automated  cockpits. 
They  claim  that  more  training  should  be  added  to  the  curricula  to  address  the 
automation-related  problems  in  order  to  reduce  the  costs  of  the  benefits. 

Another  obvious  reason  is  the  increased  workload  in  the  modern  glass 
cockpit.  Most  modern  cockpits  contain  many  more  systems  along  with  more 
sophisticated  menus  and  operational  procedures.  In  some  situations,  performing 
all  of  the  required  tasks  is  beyond  users’  capabilities.  For  instance,  most  modern 

fighter  aircraft  have  Flight  Control  Systems  containing  certain  safety  limitations; 
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they  do  not  allow  pilots  to  exceed  structural  g  limits.  The  designers  are  aware  of 
the  workload  and  stress  levels  during  air  combat,  and  because  there  is  a  great 
possibility  that  the  pilot  may  forget  to  monitor  the  g  levels  in  such  situations,  they 
automated  the  FLCS  to  limit  the  pilots  instead. 

After  the  overview  of  the  main  reasons  for  automation  in  cockpits,  it  can 
be  seen  that  increased  automation  is  inevitable  for  future  platforms.  There  is  no 
strong  debate  on  the  necessity  of  automated  systems,  but  naturally  it  has  been 
changing  the  pilots’  tasks  and  workload.  A  rough  comparison  would  be  a  shift 
from  performing  fewer  and  simpler  tasks  alone  to  performing  more  and  more 
complicated  tasks  together  with  automated  systems.  The  automated  systems 
can  be  the  autopilot  navigating  the  airplane  and  the  radar  system  performing 
locking  and  tracking  tasks,  freeing  the  pilot  for  other  imperatives. 

Sarter,  Woods  and  Billings  (1997)  said  of  the  common  role  change  in  the 
modern  cockpits  that,  “Introduction  of  new  automation  has  shifted  the  human  role 
to  one  of  monitor,  exception  handler,  and  manager  of  automated  resources”  (p. 
2).  The  same  role  shift  is  also  acknowledged  in  the  military  aviation  community, 
and  Olson  (2001)  is  one  of  the  researchers  who  agreed  that  the  shift  in  the  role 
of  pilots  with  increased  automation  will  be  from  performing  all  the  tasks  to 
supervisory  control. 

The  role  shift  in  the  cockpit  requires  pilots  to  perform  their  tasks  in  a 
different  way  than  before.  Furthermore,  they  have  to  develop  the  corresponding 
skills  and  abilities  to  be  successful  in  their  new  role.  It  is  known  that  actually 
performing  a  task  versus  observing  it  (vigilance  task)  offers  an  improvement  in 
terms  of  performance.  It  is  a  common  fact  that  humans  are  not  so  effective  in 
vigilance  tasks.  Rigner  and  Dekker  (2000)  acknowledged  the  effects  of 
automation  on  required  pilot  skills.  They  said  that  the  pilots  have  become 
supervisors  in  the  cockpit,  observing  and  maintaining  the  operation  of  all  other 
systems  and  components.  In  order  to  succeed  in  their  new  roles,  pilots  should 
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absorb  more  knowledge  and  build  many  more  complicated  skills.  Managing  their 
attentional  resources,  and  filtering  and  evaluating  required  data,  are  some  of  the 
important  skills  they  need. 

An  interesting  point  between  Sarter,  Woods  and  Billings  (1997),  and 
Rigner  and  Dekker  (2000)  is  the  need  for  cooperation  with  other  systems  or 
resources.  Sarter,  Woods  and  Billings  (1997)  confirmed  the  importance  of  the 
human-machine  coordination,  and  acknowledged  that  the  relationships  and 
reliance  factors  are  highly  complicated  in  modern  systems.  The  pilots  should 
develop  satisfactory  mental  models  about  the  automated  systems.  There  are 
many  types  of  complicated  relationships  between  the  systems  themselves  and 
between  the  pilots  and  the  automated  systems.  Pilots  need  to  know  and  direct 
the  systems  interactions,  and  be  aware  of  the  all  inputs  to  and  outputs  from  the 
autopilot  during  the  flight. 

Apparently,  the  skills  emerging  with  the  use  of  automation  are  the 
supervisory  skill  to  observe  the  automated  systems  and  the  coordination  and 
cooperation  skills  to  work  with  the  automated  systems  as  if  they  were  another 
agent  in  the  cockpit.  There  are  many  instances  proving  that  the  implementation 
of  automation  in  the  cockpits  can  be  problematic.  Olson  (2001)  reported  one  of 
the  important  findings  about  the  pilot-autopilot  coordination:  in  most  cases,  after 
the  automated  system  starts  to  perform  a  task,  the  pilots  tend  to  forget  to  control 
the  related  performance  parameters.  This  finding  directly  shows  that  the  pilots  do 
not  observe  the  automation  performance  as  they  should.  Initially,  one  can  directly 
blame  pilots,  but  this  would  be  not  the  real  solution,  but  an  escape  from  the  big 
picture.  The  problems  should  be  addressed  beginning  with  the  design  of  the 
system,  building  the  operational  procedures,  and  training  to  control  quality 
throughout  the  lifetime.  The  common  problems  of  automation  will  be  discussed  in 
the  next  chapter,  and  the  recommendations  in  the  last  one. 

To  provide  a  quick  view  of  the  big  picture  about  relations  in  the  automated 

cockpits,  Spencer  (2000)  mentioned  a  beneficial  model.  The  main  idea  of  the 

model  is  to  define  the  systems  with  the  subcomponents  of  software,  hardware, 

8 


environment  and  liveware.  The  liveware  in  the  cockpit  example  is  the  pilot,  and 
he  interacts  with  all  of  the  other  components  during  the  missions.  The  other 
components  are:  standard  operational  procedures,  the  software  of  the  pilot 
vehicle  interfaces,  the  aircraft  and  the  related  hardware,  co-pilots,  etc. 

2.  Definitions  and  Overview  of  the  Automation  Problems 

In  order  to  understand  and  interpret  the  results  of  the  research  better,  it  is 
crucial  to  acquire  enough  knowledge  about  the  terminology  and  common 
knowledge  of  related  fields.  This  is  also  applicable  to  cockpit  automation,  about 
which  there  are  several  commonly  accepted  definitions.  These  definitions,  along 
with  some  examples  will  be  reviewed  in  this  chapter  to  provide  the  fundamentals 
needed  in  the  following  sections.  Although  there  exist  many  other  phenomena 
about  cockpit  automation,  the  mentioned  ones  are  the  most  important  and 
recurring  ones  in  the  literature. 

a.  Mode  Awareness/Confusion/Errors 

The  reason  to  mention  these  three  terms  together  is  the  fact  that 
they  are  inextricably  inter-related.  They  are  used  interchangeably  in  many 
studies,  and  are  based  on  the  same  phenomenon. 

Today,  most  of  automated  systems  consist  of  many  operating 

modes  and  menus.  They  are  also  capable  of  making  many  decisions  and 

performing  tasks  independently.  As  mentioned  before,  it  will  be  legitimate  to  think 

that  there  are  two  actors  in  a  highly  automated  cockpit,  and  therefore  a  very 

logical  issue  is  the  coordination  between  them.  In  other  words,  pilots  should  be 

aware  of  what  the  other  decision  maker  is  doing  at  all  times,  and  how  it  also 

interacts  with  the  other  systems.  This  straightforward  logic  is  self-explanatory, 

and  even  if  one  does  not  have  any  knowledge  about  cockpit  automation,  he  can 

conclude  that  “mode  awareness”  means  continuously  to  be  aware  of  the  mode  in 

which  the  system  is  operating.  Similarly,  if  the  operator  is  confused  about  the 

mode  of  the  automation,  it  is  “mode  confusion;”  and  if  he  commits  errors  because 

of  this  confusion  they  are  “mode  errors.”  Even  though  they  seem  very 
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straightforward,  there  are  many  valuable  and  important  data  about  mode  errors 
in  the  literature,  and  these  will  be  discussed  later  in  this  study. 

Sarter  (2000)  is  one  of  the  researchers  who  has  studied  cockpit 
automation.  She  said  that  the  pilots  generally  do  not  have  a  through 
understanding  of  the  structure  and  operational  procedures  of  automated 
systems,  and  when  combined  with  “low  observability,”  this  creates  various 
problems  in  flight  (p.  233).  Yet  another  additional  system  is  then  implemented  to 
help  pilots,  with  yet  greater  need  for  additional  visual  and  mental  resources. 

The  reasons  for  the  mode  awareness  problems  in  Sarter’s  study 
are  the  knowledge  problems  and  low  observability;  furthermore,  Sarter,  Mumaw, 
and  Wickens  (2007)  defined  four  major  factors  causing  the  automation  problems 
in  glass  cockpits.  The  first  one  is  related  to  the  lack  of  capturing  the  attention  of 
the  pilots.  Most  automation  systems  have  poor  design  and  fail  to  provide 
sufficient  and  easily  observed  feedback  to  pilots.  Also,  the  increased  number  of 
automated  actions  shunt  the  pilots  increasingly  into  a  supervisory  role;  the 
automated  systems  initiate  more  actions  without  any  pilot  consent,  which  makes 
it  harder  to  track  the  modes.  Another  reason  is  the  “high  degree  of  system 
coupling”  (p.  348).  In  general,  not  only  the  modes  and  complexity  of  the 
automated  systems,  but  also  the  number  of  other  systems  have  increased. 
These  developments  conspired  to  hinder  the  pilots’  ability  to  track  and 
understand  the  interactions  between  the  automated  and  other  systems.  More 
systems  became  related  to  the  inputs  and  outputs  to  automation.  The  last 
contributing  factor  is  the  knowledge  of  the  pilots.  It  was  found  that  the  pilots’ 
lacked  robust  understanding  of  the  automated  systems,  and  were  prone  to  mode 
awareness  problems. 

When  the  operator  takes  an  action  as  a  result  of  thinking  a  system 
is  in  one  mode,  where  it  is  actually  in  another  mode,  this  is  a  “mode  error”  (Sarter 
etal.,  2007,  p.  347). 
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Examples  will  give  a  better  insight  on  the  mode  awareness 
concerns.  A  mode  can  be  a  navigation  mode  of  an  autopilot  system,  or  a  tracking 
mode  of  a  fighter’s  fire  control  radar.  In  both  situations,  it  is  very  crucial  to  be 
aware  of  the  mode  of  the  automated  system.  Most  of  the  evidential  accidents 
about  the  automation  can  be  found  in  the  literature.  Sarter  (2000)  mentioned  two 
of  these  incidents.  In  the  first  one,  the  captain  changed  the  descent  mode  of  the 
autopilot,  because  it  deviated  from  the  approach  track,  and  switched  to  the 
“heading  select  mode.”  Because  the  vertical  operation  part  of  the  autopilot  was 
working  with  the  lateral  part,  the  change  resulted  in  an  unintended  1800  fpm 
descent,  which  was  considerably  in  excess  of  the  proper  rate  for  landing. 
Thankfully,  the  warning  system  in  the  aircraft  and  the  crew  of  the  control  agency 
together  prevented  a  potential  accident.  In  the  second  example,  the  pilots 
thought  that  they  were  making  a  3.3  degrees  descent,  whereas  the  autopilot 
commanded  a  3300  fpm  descent;  the  crew  and  passengers  were  not  as  lucky  as 
in  the  first  example,  unfortunately,  and  the  plane  crashed.  These  real  life 
examples  indicate  the  importance  of  mode  awareness  in  autopilot  systems. 
Because  the  common  approach  is  not  to  perceive  the  autopilot  as  a  critical 
system,  the  aviation  community  must  make  sure  to  take  the  preventive 
measures. 

Although  there  are  no  examples  about  mode  confusion  in  fighter 
jets,  it  does  not  mean  that  fighters  will  not  have  those  problems.  The  main  reason 
is  the  adoption  of  automation.  Commercial  aviation  already  implemented  the 
automated  systems,  and  most  of  the  tasks  can  be  performed  with  the  autopilots. 
But  to  date,  the  fighter  cockpits  do  not  contain  the  same  degree  of  automation. 
The  autopilot  and  other  automated  systems  are  not  used  and  implemented  as  in 
the  commercial  aviation  community.  But  this  trend  is  changing,  as  the  authors 
observe  in  the  JSF  example.  Not  just  the  autopilot,  but  also  even  the  auto-mode 
radar  may  cause  problems  with  mode  confusion.  For  instance,  if  the  digit  on  one 
side  of  a  radar  target  symbology  represents  the  altitude  in  one  mode,  but  another 
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data  in  another  mode,  the  pilot  may  build  an  intercept  strategy  depending  on  the 
wrong  interpretation  of  the  data,  which  in  turn  may  mean  life  or  death. 


b.  Decompensation  Incident,  Automation  Omission  and 
Commission  Errors 

The  “decompensation  incident”  occurs  when  automation  tries  to 
handle  some  problem  or  abnormal  parameter  up  to  a  point,  and  then  quits 
handling  that  problem.  In  such  situations,  pilots  are  liable  to  become  aware  of  the 
problem  too  late  (Sarter  et  al.,  1997,  p.  7).  This  definition  is  similar  to  the 
definition  of  the  “automation  omission  error.”  Mosier,  Skitka,  Heers,  and  Burdick 
(1998)  explained  the  automation  omission  error  as  an  error  caused  by  a  lack  of 
correct  information  about  an  abnormal  situation  during  automated  operations  (p. 
51).  The  example  they  give  is  an  airline  mishap  in  1996.  The  airplane  was 
controlled  by  the  autopilot  for  level  flight.  One  of  the  aircraft  engines  failed,  but 
the  autopilot  compensated  for  the  loss  of  thrust  caused  by  that  engine,  and  as 
the  aircraft  continued  to  fly  level,  the  pilots  did  not  understand  the  abnormal 
situation.  Once  they  disengaged  the  autopilot,  however,  the  airplane  entered  into 
an  abnormal  descent. 

Mosier  et  al.  (1998)  also  explained  the  automation  commission 
errors.  These  errors  happen  when  pilots  monitor  contradicting  behaviors  between 
other  and  automated  systems.  The  example  they  mention  from  the  literature  is  a 
study  with  an  experimental  design.  The  pilots  were  given  a  scenario  with 
contradicting  cues.  During  takeoff,  the  automatic  checklist  directed  the  pilots  to 
shutdown  an  engine,  though  the  truth  as  shown  by  traditional  displays  was 
different.  The  other  engine  was  malfunctioning,  but  75%  of  the  participants  took 
improper  action  and  trusted  the  automated  system  without  cross  checking  it  with 
traditional  instruments.  Both  types  of  errors  happen  because  of  the  lack  of  cross 
checking  of  other  flight  instruments  with  the  automated  ones.  This  finding, 
combined  with  the  finding  that  pilots  do  not  include  status  checks  of  automated 
systems  in  their  routine  checks  (as  they  did  with  other  systems  and  aircraft 
performance  variables)  indicate  a  problem  in  automation  management. 
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c.  Clumsy  Automation 

One  of  the  major  promises  of  automation  was  reduced  pilot 
workload — because  the  pilot  would  perform  fewer  tasks  due  to  automation,  the 
overall  workload  could  be  expected  to  decrease.  This  is  a  topic  for  debate,  and 
Sarter  et  al.  (1997)  explained  the  definition  of  “clumsy  automation”  by  Wiener 
(1989).  They  explained  that  “many  automated  systems  support  pilots  most  in 
traditionally  low  workload  phases  of  flight  but  are  of  no  use  or  even  get  in  their 
way  when  help  is  needed  most,  namely  in  time-critical  highly  dynamic 
circumstances”  (p.  3).  This  will  quite  probably  be  the  case  in  the  new  fighter 
cockpit  environment.  The  intent  of  the  authors  is  not  to  say  that  the  fighter 
cockpits  have  clumsy  automation,  but  that  the  environment  is  generally  as 
mentioned  above:  timely,  critical  and  highly  dynamic  (Sarter  et  al.,  1997). 
Another  important  change  in  the  fighter  cockpits  is  the  nature  of  the  missions. 
The  new  missions  require  many  demanding  and  dynamic  tasks  such  as  air 
combat,  SEAD,  and  DEAD  missions,  whereas  the  navigation  is  the  only  task  in 
commercial  aviation.  This  concern  will  also  be  discussed  in  the  following 
chapters. 

In  addition  to  the  aforementioned  problems,  Doherty  (2001) 
reported  “boredom”  and  “complacency”  (p.  22).  Because  the  pilots  feel  that  the 
reliance  on  them  by  the  systems  is  decreased  due  to  automation,  their 
performance  tends  to  decrease  with  the  automation-related  tasks.  The  effect  of 
boredom  is  that  it  causes  the  stress  level  to  be  lower  than  the  optimum,  and  the 
effect  of  complacency  is  the  increased  demand  for  cognitive  sources.  The 
boredom  can  cause  a  decrease  in  pilots’  SA,  whereas  the  complacency  can 
cause  task  saturation  under  highly  dynamic  and  high  workload  situations. 

The  main  objective  of  this  section  was  to  provide  the  readers  with 
fundamental  knowledge  about  the  human  factors  concerns  about  cockpit 
automation.  The  findings  will  be  discussed  further  in  the  next  sections. 
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3.  Findings  About  Cockpit  Design 

Inevitably,  as  more  and  more  technological  systems  are  used  in  the 
cockpit,  many  human  factors  concerns  accompanied  them.  Automation  is  one  of 
the  major  evolutions  among  the  modern  technological  systems  in  glass  cockpits. 
It  has  two  aspects:  hardware  and  software.  In  some  instances,  pre-existing 
systems  have  been  digitized,  and  algorithms  have  been  specifically  coded  to 
automate  many  tasks.  In  other  cases,  hardware  is  designed  specifically  to 
automate  some  tasks,  such  as  autopilots.  Regardless  of  the  method,  the  process 
has  had  a  major  impact  on  human  factors  in  cockpit  design. 

Spencer  (2000)  explained  the  environment  in  modern  glass  cockpits  as 
“Complex  cockpits,  faster,  more  capable  aircraft,  airspace  saturation,  and 
increasing  air  traffic  control  requirements  create  the  environment  and  conditions 
conductive  to  mode  confusion”  (p.  vi). 

The  reason  to  include  this  chapter  is  neither  to  defend  nor  to  criticize  the 
design  of  the  JSF  cockpit,  but  to  identify  the  potentially  problematic  areas.  One  of 
the  commonly  accepted  impacts  is  the  shift  in  the  pilot’s  role  in  the  modern 
cockpit.  This  is  a  topic  studied  thoroughly  under  CRM  and  other  automation 
research.  Spencer’s  model  (described  previously)  for  the  interactions  between 
pilot  and  components  in  the  cockpit  treats  the  pilot  as  essentially  another  system. 
This  systems  approach  may  provide  designers  a  framework  to  consider  the 
relations  in  the  cockpit  because,  as  he  notes,  the  tendency  with  modern  cockpit 
design  is  to  prioritize  the  automated  systems,  whereas  a  better  way  is  to  use  a 
human-centered  approach  (Spencer,  2000).  As  seen  also  in  many  other  studies, 
the  human  has  not  been  the  major  concern  in  modern  cockpit  design.  However, 
the  pilot  is  still  responsible  for  safe  and  efficient  operation  of  the  aircraft.  Some 
steps  forward  can  be  seen  in  many  cockpit  designs,  but  there  is  room  for 
improvement  on  this  issue. 

Likewise,  another  consequence  of  the  automated  cockpits  is  the  shift  in 
the  type  of  activities  conducted  by  pilots.  They  are  not  actively  performing,  but 
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observing  the  automated  tasks,  and  it  is  commonly  accepted  that  humans  cannot 
maintain  their  performance  during  longer  periods  of  vigilance  tasks.  Davies  and 
Parasuraman  (1982)  are  among  the  researchers  who  studied  this  phenomenon 
by  questioning  the  findings’  relevance  in  real  conditions.  They  explain  the 
shortcomings  of  some  findings  of  fully  laboratory  experiments,  but  also  agree 
with  the  decreasing  performance  over  time  shown  in  many  other  studies.  With 
the  tasks  the  autopilot  performs,  and  the  pilot  observes,  there  can  be 
shortcomings  due  to  vigilance.  That  raises  the  question  of  the  number  of  tasks  to 
be  automated  during  design.  Spencer  (2000)  reported  the  same  issue,  and  noted 
that  there  is  a  threshold  for  the  best  performance  in  automated  cockpits. 
Basically,  up  to  this  threshold  more  automation  results  in  better  performance.  But 
after  this  point,  if  additional  tasks  are  also  automated,  the  pilot  performance 
suffers  due  to  SA  problems.  He  added  that  “Researchers  recommend  balancing 
human  involvement  (increased  situational  awareness)  with  automation  efficiency” 
(p.  12).  This  conclusion  may  be  valid  in  general,  but  establishing  a  balance  will 
be  harder  in  dynamic  situations.  All  of  a  sudden,  additional  threats  may  pop  up, 
thus  shifting  the  balance  point  for  that  situation.  With  these  challenges  in  mind, 
the  human  involvement  should  at  least  be  taken  into  consideration  during  design 
and  operational  phases. 

Since  more  systems  are  in  use  in  modern  cockpits,  the  users  have  to  do 
more  with  their  same  cognitive  resources.  In  general,  mental  resources  are  a 
concern  for  pilot  workload  studies,  but  the  over-reliance  on  the  visual  resource  is 
becoming  a  more  important  concern.  The  pilots  monitor  basic  flight  instruments, 
navigational  and  communicational  systems,  Air  to  Air  and  Air  to  Ground  Radars, 
Radar  Warning  Receivers,  automated  systems  and  many  more,  and  they  use 
their  visual  system  for  all  of  these  tasks.  Increasingly,  the  question  is  whether  the 
pilots’  visual  resources  are  able  to  handle  this  workload  in  all  of  the  required 
situations  in  the  cockpit.  In  fact,  the  automated  systems  also  demand  visual 
resources  in  many  applications,  adding  yet  another  system  to  help  pilots  but 
contributing  to  a  need  for  additional  visual  resources. 


15 


Sarter  (2000)  tried  to  come  up  with  solutions  for  decreasing  the  attention 
load  on  visual  resources.  In  order  to  mitigate  the  problems  with  automation,  one 
has  to  understand  the  reasons  for  the  problems  during  automation  use.  Sarter 
(2000)  reported  one  of  the  findings  of  the  study  by  Sarter  and  Woods  (1995), 
where  the  pilots  did  not  implement  the  automated  systems  checks  to  their  routine 
procedures,  but  mostly  observed  them  depending  on  their  “expectations  of 
specific  automation  behavior”  (Sarter,  2000,  p.  234).  She  concluded  that  this 
approach  will  fail  if  the  pilot  is  not  fully  aware  of  the  status  and  modes  of  the 
autopilots  (Sarter,  2000).  The  reason  may  not  be  the  overload  of  visual 
resources,  but  additional  resources  can  be  addressed  in  order  to  provide  pilots 
with  increased  awareness  about  the  automated  systems. 

Another  issue  Sarter  (2000)  noted  is  the  problems  in  display  designs  of 
automated  systems.  The  modern  trend  has  been  to  convert  the  round  shaped 
analog  displays  to  “so-called  tape  instruments”  (p.  236),  but  the  pilots,  at  a 
glance  were  able  to  retrieve  the  required  information  from  the  round  displays. 
She  reported  one  of  the  findings  in  a  study  of  Sarter  and  Woods  (1995b),  that 
pilots  expressed  difficulties  about  the  non-round  displays.  They  added  that  it  took 
longer  to  get  the  data  from  tape  displays.  That  finding  points  to  the  issue  of 
additional  cognitive  processes  and  the  increased  demand  on  visual  resources. 
This  issue  is  important  even  if  the  automated  systems  do  not  have  tape  displays. 
The  common  use  of  tape  displays  is  in  flight  performance  parameters,  and  if  the 
autopilot  is  controlling  the  airplane,  those  displays  become  one  of  the  crosscheck 
displays  to  the  pilots  monitoring  the  automated  systems. 

Another  automation  aspect  of  cockpit  design  is  the  reliance  on  the 
automated  systems  and  their  programmed  algorithms.  Automation  is  intended  to 
provide  safe  and  efficient  operations,  but  there  remains  the  risk  of  relying  on  the 
technology  too  much.  Dalcher  (2007)  studied  this  issue  by  examining  a  case  in 
his  research  where  the  autopilot  limited  the  pilot  to  an  extent  that  he  couldn’t 
recover  the  aircraft;  in  that  case,  the  automation  itself  caused  the  aircraft  to 
crash.  Dalcher  believes  that  the  goals  with  the  highly  automated  systems  are 
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the  superior  computational  capability  coupled  with  elimination  of  human  error, 
and  the  reduction  in  work  overload  and  lack  of  dependability,”  and  then 
acknowledges  that  there  may  be  many  situations  where  the  automated  systems 
lack  the  flexibility  to  handle  difficult  situations  (p.  352).  This  is  related  to  the 
algorithms  and  design  of  the  automated  systems.  Many  tasks  are  automated,  but 
sometimes  it  may  not  be  possible  to  consider  all  of  the  possible  situations  or 
inputs  and  outputs  when  designing  the  responses  of  the  automated  systems. 
Thus,  the  designers  should  be  very  cautious  while  choosing  and  implementing 
the  tasks  to  be  automated. 

After  expressing  the  importance  of  involving  users  in  the  design  and 
operational  considerations,  looking  to  the  user  side  of  automation  design  will 
provide  valuable  clues.  Tenney,  Rogers  and  Pew  (1998)  conducted  a  study  via 
survey  methodology,  and  tried  to  capture  the  pilots’  opinions  about  cockpit 
automation.  The  following  findings  are  listed  in  their  study.  First,  pilots’ 
preference  was  that  the  automation  should  be  a  cooperative  design,  rather  than 
highly  automated  cockpits  that  make  pilots  mostly  observers.  Pilots  want 
automation  in  their  complex  tasks,  but  they  desire  a  human-centered  approach, 
and  cooperation  among  the  agents  in  the  cockpit.  However,  they  reported  some 
modes  of  automation  as  “...producing  a  high  mental  workload.”  The  pilots  found 
the  most  important  features  to  be  “simplicity  and  reliability”  (p.  103).  The  reliability 
is  more  related  to  design  issues,  and  must  be  considered  from  the  conceptual 
design  phases  to  the  end  of  the  life  cycle  of  the  products.  But  it  is  not  planned 
and  implemented  by  the  users.  This  is  not  the  case  in  simplicity.  Simplicity  is  a 
usability  issue,  and  the  user  will  be  directly  affected  by  it.  Simple  systems  will 
decrease  the  workload  of  the  operators,  and  free  time  for  other  tasks. 
Furthermore,  users  will  be  prone  to  fewer  failures  with  easy-to-use  systems.  It  is 
very  apparent  that  the  pilot  opinions  basically  confirm  the  findings  of  the 
aforementioned  studies,  further  validating  those  findings. 

The  types  of  automation  play  a  great  role  in  assessing  the  attentional 
pattern  of  pilots  and  their  mental  approach  to  automation.  Wickens  (2000) 
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divided  automation  into  two  main  groups:  “reliable”  and  “unreliable.”  He  defines 
reliable  automation  as  ’’that  which  does  what  it  was  intended  to  do”  (p.  4),  and 
suggests  that  reliable  automation  causes  either  “attentional  tunneling”  or 
“cognitive  tunneling,”  depending  on  the  cognitive  process. 

The  same  phenomenon  is  named  by  Wickens  (2000)  as  “action  tunneling” 
if  it  happens  following  cognitive  tasks  about  automation,  since  the  automation  is 
naturally  going  to  direct  the  pilot’s  attention  to  the  automation’s  preferred  choice 
of  action.  Wickens  (2000)  explains  this  term  as  an  alternative  action  preferred  by 
automation  rather  than  the  pilot,  which,  however,  is  not  the  best  alternative  at  the 
moment  because  of  considerations  that  are  not  related  to  automation.  In  this 
case,  instead  of  devising  an  action  himself,  the  pilot’s  attention  is  going  to  be 
allocated  to  validating  the  automation’s  recommended  action. 

Supporting  this  suggestion,  Wickens  (2000)  observed  this  phenomenon  in 
an  experiment  in  which  pilots  did  the  pre-flight  check  with  wearable-computers. 
The  experiment  results  showed  that  the  detection  of  unidentified  faults  decreased 
while  the  detection  rate  of  computer-identified  faults  increased.  The  information 
that  is  considered  to  be  significant  and  is  highlighted  by  the  automation  might 
cause  the  pilot  to  ignore  other  information  that  is  not  highlighted  at  the  moment, 
but  is  as  critical  as  the  highlighted  one. 

Eventually,  says  Wickens,  the  pilot  will  begin  to  trust  in  automation  more 
than  in  himself;  he  will  not  question  the  validity  of  automation-inferred  data  and 
actions  as  much  as  he  should.  This  will  lead  to  the  failure  to  detect  an  automation 
problem,  degrade  the  level  of  situational  awareness  of  the  pilot,  and  cause  him  to 
lose  his  skills  that  have  been  replaced  by  automation.  The  loss  of  situational 
awareness  due  to  the  loss  of  automation  will  cause  the  pilot  to  evaluate  the  raw 
data  and  the  systems  outputs  either  slowly  or  not  at  all.  In  any  case,  the  pilot  will 
fail  to  consider  all  possible  alternative  actions  due  to  this  deprived  situational 
awareness  caused  by  “automation  complacency.” 
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4.  Findings  About  Pilots’  Strategies  of  Automation  Management 

Sarter  and  Woods  (1992)  mention  the  commonly  accepted  perspective  of 
Billings  (1991),  that  the  job  of  the  pilots  has  shifted  to  that  of  a  supervisor  only 
actually  performing  in  abnormal  procedures;  at  the  same  time,  they  also  discuss 
that  taking  the  human  user  out  of  the  loop  is  a  risky  decision  that  will  affect  the 
concentration  and  “awareness”  of  the  pilots  negatively. 

Sarter  and  Woods  (1992)  conducted  a  study  in  1992.  They  report  that 
even  the  pilots  who  used  the  automated  systems  more  than  one  year  sometimes 
had  difficulties  operating  and  understanding  the  automated  systems,  and  they 
recommended  that  enough  time  should  be  given  during  pilot  training  to  help  them 
build  a  through  understanding  of  the  automated  systems,  and  also  to  address  the 
pilot-autopilot  “coordination”  during  training.  This  was  one  of  the  earlier  studies, 
and  pointed  out  an  important  problem  about  the  pilot-autopilot  interaction.  It  is 
legitimate  to  think  that  the  automated  systems  were  not  as  complex  then  as  they 
are  today;  thus,  even  with  less  complex  systems  offered  the  same  problems.  But 
another  factor  could  be  the  fact  that  automation  was  in  its  earlier  stages,  and 
therefore  that  robust  training  and  operational  procedures  had  not  been  fully 
developed.  So,  reviewing  further  studies  may  make  the  subject  more  clear. 

Sarter  and  Woods  (1994)  also  conducted  an  objective  study,  with  a  typical 
flight  scenario  from  LAX  to  SFO  with  a  part  task  trainer.  They  injected  events 
related  to  automation  during  flight,  and  recorded  pilots’  reactions  to  a  prepared 
list.  Using  this  method,  they  also  asked  participants  questions  about  automated 
systems  during  low  workload  conditions.  They  tried  to  capture  the  pilots’ 
knowledge  level  in  a  realistic  environment  rather  than  in  isolated  situations.  They 
report  that  pilots  were  comfortable  in  performing  the  standard  basic  tasks  in 
cockpits  such  as  “intercepting  a  radial,  building  or  executing  a  holding  pattern...,” 
but  they  experienced  difficulties  during  the  tasks  requiring  comprehensive 
understanding  about  the  automation  such  as  “aborting  a  takeoff  at  40  knots  with 
auto-throttles  on“  and  “...anticipating  when  go-around  mode  becomes  armed 

through  landing....,”  both  indicating  that  the  pilots  have  insufficient  knowledge 
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about  the  automated  systems,  even  though  they  think  that  they  have  enough 
knowledge  (p.  14).  The  interesting  point  is  the  lack  of  knowledge  and  operational 
capabilities  about  automation  even  in  some  emergency  situations.  Even  if  the 
autopilot  may  not  seem  complex  and  important,  the  mishaps  and  crashes 
express  its  importance. 

Based  on  the  findings,  Sarter  and  Woods  (1994)  suggest  improved 
training  to  address  developing  automation  skills.  The  pilots  not  only  have  to 
understand  the  technical  structure  of  the  systems,  but  also  be  able  to  make 
decisions  under  high  workload  conditions.  This  requires  a  thorough 
understanding  of  the  functions  and  interrelations  of  the  automated  systems.  This 
is  an  issue  to  be  addressed  properly  in  realistic  environments,  via  scenarios.  The 
knowledge  and  operational  abilities  of  pilots  should  be  stressed  with  these 
scenarios  in  order  to  help  them  build  proper  mental  models. 

Sarter,  Woods  and  Billings  (1997)  claim  that  newer  approaches  in 
automation  training  should  not  only  include  increasing  the  training  time,  but  also 
modifying  the  nature  of  the  content.  The  two  important  traps  about  automation 
management  reported  in  their  findings  are  the  subjective  opinion  of  the  pilots  that 
they  have  enough  capabilities  and  knowledge,  and  their  tendency  to  build  only 
limited  skills  that  they  try  to  apply  to  all  situations.  These  two  points  mask  the 
truth  for  pilots,  and  in  case  of  a  life  or  death  situation,  it  may  be  too  late  to 
discover  their  deficiencies  in  automation. 

Mosier,  Skitka,  Heers,  and  Burdick  (1998)  investigated  “automation  bias” 
during  automated  tasks  performed  by  commercial  pilots.  They  conducted  an 
experiment  measuring  the  “automation  omission,”  and  “automation  commission” 
errors  of  the  participants  during  automated  tasks.  For  instance,  commission 
errors  happen  when  pilots  monitor  contradicting  behaviors  between  traditional 
and  automated  systems.  The  example  they  give  is  when  pilots  were  given  a 
scenario  with  contradicting  cues.  During  takeoff,  the  automatic  checklist  directed 
the  pilots  to  shutdown  an  engine;  the  truth  as  shown  in  traditional  displays  was 
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different.  The  other  engine  was  malfunctioning,  but  75%  of  the  participants  took 
improper  action  and  trusted  the  automated  system  without  cross  checking  it  with 
traditional  instruments. 

Both  omission  and  commission  errors  happen  because  of  the  lack  of  cross 
check  of  traditional  flight  instruments  along  with  automated  ones.  This  finding, 
combined  with  the  finding  that  pilots  do  not  include  a  routine  status  check  of 
automated  systems  as  they  do  with  traditional  systems  and  aircraft  performance 
variables,  again  indicates  the  problem  of  automation  management.  Additionally 
another  interesting  finding  of  their  study  is:  “...pilots  who  reported  a  higher 
internalized  sense  of  accountability  for  their  interactions  with  automation  verified 
correct  automation  functioning  more  often  and  committed  fewer  errors  than  other 
pilots”  (Mosier  et  al. ,  1998,  p.  59). 

Another  study  that  reports  similar  findings  was  conducted  by  Bjorklund, 
Alfredson,  and  Dekker  (2006).  They  conducted  an  experiment  to  study  the 
relation  between  the  monitoring  strategies  of  the  pilots  and  mode  awareness. 
They  collected  eye-tracking  data,  while  giving  participants  mode  transition  and 
asking  for  status  of  autopilot  mode.  The  results  supported  previous  findings  that 
pilots  did  not  follow  the  standard  operating  procedures,  but  pursued  their  own 
strategies  driven  by  expectations  to  monitor  the  status  of  autopilot.  They  report 
that  the  pilots  did  not  capture  40%  of  the  mode  transitions.  The  pilots  again  did 
not  monitor  the  autopilot  via  the  SOP,  and  during  their  routine  checks..  This  study 
was  conducted  in  2006,  and  the  findings  are  still  similar  to  the  studies  done  in  the 
early  1990s.  With  time,  one  expects  that  the  training  and  operational  procedures 
will  have  addressed  the  common  problems  about  automation,  because  it  has 
been  widely  used  for  decades.  However,  studies  indicate  the  existence  of  the 
same  problems.  Additionally,  this  is  not  the  only  recent  study  indicating  the  same 
results. 

Sarter,  Mumaw,  and  Wickens  (2007)  conducted  a  study  researching  the 

same  issues.  In  their  study,  they  mention  the  previous  studies  and  point  out  that 

most  of  them  used  subjective  measures  about  the  pilots  using  the  automated 
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systems.  One  of  the  shortcomings  of  previous  studies  using  objective  measures 
was  the  small  sample  size.  In  their  study,  they  had  a  sample  size  of  20,  and  used 
a  high  fidelity  part  task  trainer  suitable  for  their  study  purpose.  They  had  two 
types  of  events,  experimentally  injected  and  others.  Other  type  of  events 
occurred  during  the  flow  of  the  experiments  themselves,  and  were  also  recorded 
for  analysis.  They  also  recorded  the  eye  movements  besides  asking  questions  to 
capture  the  level  of  pilots’  understanding  about  automated  systems.  The  actions 
taken  by  the  pilots  during  the  experiments  were  recorded  with  predetermined 
data  sheets.  Their  findings  are  consistent  with  the  previous  studies.  The  pilots 
allocated  considerably  more  of  their  attentional  resources  on  basic  flight 
parameters  than  the  automated  systems.  Their  automation  awareness  was  much 
lower  than  their  general  awareness,  and  one  of  the  reasons  is  that  they  failed  to 
build  robust  mental  medals  about  the  automation  (Sarter,  Mumaw,  &  Wickens, 
2007). 

B.  ALLOCATION  OF  ATTENTION,  WORKLOAD  AND  TASK 

MANAGEMENT 

Doherty  (2001)  pointed  out  the  most  important  and  obvious  difference 
between  commercial  and  military  aviation  is  that  the  mission  in  military  aviation 
goes  beyond  basic  navigation.  In  addition,  the  real  missions  (intercepts,  target 
bombing,  Air  Combat  Maneuvers)  are  as  not  static  as  level  flight  in  commercial 
aviation;  thus,  the  workload  and  management  of  attentional  resources  are  more 
demanding  in  military  aviation. 

1.  Attention  Theory 

Fleetwood  and  Byrne  (2004)  defined  attention  as  a  resource,  and  the  way 
it  is  allocated  is  important  because  of  limited  resources. 

Prinzel  and  Risser  (2004)  pointed  out  the  three  sub  categories  of 
attention,  which  are  more  dominant  in  aviation  and  Head-up  Display  (HUD) 
literature,  as  selective,  focused  and  divided  attention  modes.  (Prinzel  III  &  Risser, 
2004) 
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It  is  beneficial  at  this  point  to  examine  focused  and  divided  attention 
modes  a  little  bit  more,  to  have  a  better  insight  about  accidents.  Prinzel  and 
Risser  (2004)  examined  these  attention  modes  with  space-  and  object-based 
theories.  Space-based  theory  considers  the  distance  between  information 
sources,  and  suggests  that  the  more  these  sources  are  away  from  each  other  the 
more  eye  movements  will  be  required  for  scanning,  thus  resulting  in  a 
performance  degradation  due  to  increased  scanning  costs.  It  favors  the 
information  sources  that  are  close  to  each  other  for  attention  to  support  the 
concurrent  tasks  requiring  information  from  these  sources.  This  theory,  however, 
also  considers  the  cluster  effects;  it  will  be  possible  to  give  equal  attention  to  two 
sources  if  they  are  in  “close  spatial  proximity,”  according  to  Prinzel  and  Risser 
(2004).  On  the  other  hand,  they  claim  object-based  theories  suggest  that 
attention  supports  concurrent  processing  of  tasks  using  information  from  sources 
that  belong  to  the  same  object,  defying  spatial  proximity. 

Besides  these,  Prinzel  and  Risser  (2004)  also  stressed  the  importance  of 
different  domains  of  information  on  attention  capture.  They  state  that  processing 
of  HUD  facilitates  a  combination  of  these  attentional  theories  and  information 
domains.  The  near  and  far  domain  each  require  focused  attention  to  extract 
information  when  switching  between  the  domains,  while  following  a  symbology 
that  is  superimposed  on  a  target  (far  domain)  on  the  HUD  requires  divided 
attention. 

Prinzel  and  Risser  (2004)  found  that  the  presence  of  more  symbology 
than  is  required  for  the  current  task  was  a  factor  making  the  attentional  switching 
harder  between  near  and  far  domains,  due  to  increased  clutter  on  the  display, 
thus  resulting  in  attentional  capture  or  tunneling. 

They  further  suggested  that  the  redundancy  of  the  information  might  be 
another  reason  for  attentional  tunneling.  Evidence  for  this  is  that  pilots  extract  the 
flight  information  only  from  the  HUD  because  it  provides  more  sensitive 
information,  and  use  the  environmental  cues  just  for  monitoring. 
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Another  possible  reason  for  attentional  tunneling  suggested  by  Prinzel  and 
Risser  is  perceptual  load.  They  stated  that  under  increased  workload  conditions, 
it  is  more  likely  to  observe  attentional  tunneling  because  pilots  cannot  scan  and 
filter  the  unrelated  and  unnecessary  data  because  of  the  high  demands  from 
attentional  sources. 

Prinzel  III  and  Risser  reported  that  “perceptual  groupings”  might  be 
another  reason  for  attentional  tunneling.  They  pointed  out  evidence  that  pilots 
group  informational  sources  according  to  their  domains.  This  grouping  is  done 
perceptually,  with  near  domain  objects  being  stationary  and  far  domain  objects 
being  in  motion.  Thus,  this  might  be  another  explanation  for  why  the  far  domain 
unexpected  events  are  hard  to  detect.  Pilots  have  difficulty  in  switching  between 
these  perceptual  groupings. 

2.  Attentional  Resources  Considerations  in  Automated  Tasks 

Wickens  (2000)  referred  to  the  following  model  to  classify  the  data 
process  stages  in  automation  (Figure  1 ). 
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Figure  1.  The  parts  of  automation  processing  (From:  Wickens,  2000,  p.  2) 
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A  thorough  understanding  of  Figure  1  is  necessary  to  understand  the 
underlying  reasons  for  attentional  tunneling,  and  the  tight  relationship  between 
automation  and  use  of  attentional  sources. 

This  figure  is  also  significant  to  observe  the  way  in  which  the  factors 
affecting  the  allocation  of  attention  implemented  in  the  information  progression 
process.  It  might  also  give  a  basic  understanding  about  the  attentional  patterns  in 
an  information  rich  environment.  Therefore,  it  is  considered  to  be  beneficial  to 
give  a  brief  explanation  of  automation,  and  a  description  of  this  figure. 

Wickens  (2000)  stated  that,  in  the  first  stage,  incoming  raw  data  might  be 
filtered  by  the  system  or  selected  to  be  in  a  certain  form  by  the  pilot.  This  raw 
data  is  represented  to  the  pilot  either  explicitly  or  implicitly  in  many  other  forms, 
once  automation  determines  it.  All  of  these  forms  have  the  same  effect  on  a 
pilot’s  attention  allocation  process:  distracting  him  from  the  other  sources  of 
information,  they  direct  a  pilot’s  attention  to  a  single  point. 

On  the  other  hand,  Funk,  Suroteguh,  Wilson,  and  Lyall  (1998)  noted  that, 
in  automation  and  attention  relationships:  “The  attentional  demands  of  pilot- 
automation  interaction  may  significantly  interfere  with  performance  of  safety- 
critical  tasks  (e.g.,  “head  down  time”,  distractions,  etc.)”  (para.  18). 

3.  SA,  Workload  and  Task  Management 

Endsley  (1995)  referred  to  her  previous  researches  while  providing  the 
definition  of  SA  as:  “Situation  awareness  is  the  perception  of  elements  in  the 
environment  within  a  volume  of  time  and  space,  the  comprehension  of  their 
meaning,  and  the  projection  of  their  status  in  the  near  future”  (p.  36). 

Doherty  (2001)  is  one  of  the  researchers  acknowledging  the  additional 
required  abilities  for  pilots  in  glass  cockpits.  One  good  example  is  the  Distributed 
Aperture  System  in  JSF.  It  includes  several  cameras  all  around  the  aircraft  and 
the  pilots  are  able  to  “see  through”  the  fuselage  in  all  directions.  The  DAS  system 
may  be  prone  to  disorientation,  which  needs  to  be  addressed  in  training.  Doherty 
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(2001)  also  states  that  the  inability  to  manage  the  ample  data  is  the  main  reason 
for  the  “excessive  workload  errors”  (p.  24).  The  pilots  in  JSF  cockpit  may  be 
prone  to  this  type  of  errors  if  they  do  not  acquire  the  skills  for  managing  the 
information  in  the  portals  in  order  to  make  their  tactical  decisions. 

In  another  study  related  to  the  same  issue,  Spencer  (2000)  reports  the 
importance  of  the  time  and  informational  overload  factors  in  user  performance, 
and  adds  that  a  University  of  Toronto  study  concluded  that  time  pressure  was 
more  important  than  informational  overload.  Although  it  seems  that  these  two 
factors  are  different,  the  distinction  may  not  be  so  clear  in  real  life.  In  many 
situations,  higher  informational  overloads  will  make  the  pilots’  job  harder,  and 
they  may  need  more  time  to  filter  and  evaluate  the  data,  and  then  make  proper 
decisions.  If  one  needs  to  study  the  effects  of  both  factors  in  such  situations,  it 
will  be  hard  to  distinguish  the  effect  of  time  pressure  from  the  informational 
overload.  Either  way,  there  remains  a  common  outcome  about  the  effects  of 
both,  and  there  is  the  need  to  address  the  related  problems  during  training,  and 
to  improve  the  pilots’  required  skills  for  modern  glass  cockpits. 

Many  studies  point  to  similar  concerns,  and  have  more  or  less  similar 
conclusions  about  the  changes  in  workload  in  modern  glass  cockpits.  As  a 
sensible  general  approach,  Fiduccia  et  al.  (2003)  proposed  a  taxonomy  of  pilot 
tasks,  dividing  them  into  skills  for  flying  the  “Physical  Airplane,”  and  those  for 
flying  the  “Mental  Airplane.”  Generally,  the  former  refers  to  aviating  or  basically 
flying  the  body  of  the  aircraft,  and  the  latter  to  operating  all  avionics,  thus  making 
tactical  and  complex  mission  related  decisions.  Fiduccia  and  his  colleagues  were 
in  a  team  studying  the  human  factors  concerns  in  general  aviation.  Following  the 
aforementioned  taxonomy,  they  pointed  out  that  the  accidents  related  to  modern 
systems  fall  into  the  “Mental  Airplane”  category,  and  that  addressing  this  issue  in 
training  by  means  of  hard,  realistic  systems  scenarios  should  be  mandatory. 
Coping  with  the  information  from  all  of  the  systems,  and  complicated  mission 
types,  the  authors  predict  that  JSF  tasks  will  increasingly  deal  with  flying  the 
“Mental  Airplane.” 
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Olson  (2001)  states  that  workload  management  should  be  addressed 
during  high  workload  situations.  In  operational  procedures  and  checklist  items, 
enough  time  should  be  given  for  the  pilots  to  activate  and  observe  the  automated 
functions.  There  might  be  two  major  aspects  of  automation  with  JSF.  The  first 
situation  is  when  the  autopilot  commands  the  aircraft  during  high  workload 
tactical  mission  phases.  These  phases  are  very  dynamic,  and  the  workload  is 
generally  very  high.  The  pilot  can  easily  focus  only  on  the  tactical  displays  and 
forget  to  observe  the  parameters  managed  by  the  autopilot  at  that  time.  Another 
issue  is  the  dynamic  changes  in  goals.  The  pilot  can  engage  the  autopilot  during 
a  Combat  Air  Patrol  mission,  and  the  goal  at  initiation  can  be  maintaining  a 
particular  airspeed  and  a  predefined  track.  But  if  the  pilot  suddenly  sees  an 
immediate  threat,  his  new  goal  could  include  increasing  the  airspeed  and 
performing  high  ‘g’  maneuvers.  When  transitioning  to  manual  control,  the  pilot 
has  to  be  aware  of  the  current  parameters  and  limitations  of  the  automated 
system. 

Funk,  Suroteguh,  Wilson,  and  Lyall  (1998)  suggest  that  task  management 
in  advanced  and  highly  automated  cockpits  may  be  problematic.  They  cite  an 
incident  report  (#92507)  from  the  Aviation  Safety  Reporting  System  (ASRS) 
database  that  indicates  the  significance  of  task  management.  In  this  incident, 
pilots  of  an  airliner  were  communicating  with  passengers  as  well  as  making  in¬ 
cockpit  communications.  Meanwhile,  the  autopilot’s  target  altitude  changed  from 
35K  to  33K  without  any  interaction.  By  the  time  the  pilots  recognized  this,  the 
altitude  was  400  feet  less  than  the  cleared  altitude,  fortunately  before  any 
accident. 

lani  and  Wickens  (2004)  suggested  a  task  hierarchy  to  ease  the  allocation 
of  attention  strategy  of  pilots  under  such  circumstances.  This  hierarchy  is  “aviate, 
navigate,  communicate  and  system  management  (ANCS)”  (p.  2). 

The  previously  mentioned  ASRS  report  is  a  good  example  of  why  task 

management  is  highly  significant  in  advanced  cockpit  aircraft.  Actual  aviation  is 

always  the  highest  priority  task  when  flying,  followed  in  succession  by  the  other 
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tasks.  The  incident  cited  above  is  another  example  of  automation  complacency. 
Over  time,  pilots  become  too  dependent  on  automation,  as  well  as  being  over 
confident  about  the  trustworthiness  of  the  automated  systems.  These  kinds  of 
mistakes  may  be  critical  and  safety  threatening,  especially  under  high  workload 
situations,  in  which  it  might  take  more  time  to  recognize  an  automation  error,  if  it 
could  be  recognized  at  all. 

Task  management  is  defined  by  Funk,  Suroteguh,  Wilson,  and  Lyall 
(1998)  as  “the  process  by  which  the  operators  of  complex  systems  prioritize  and 
perform  the  multiple,  concurrent  tasks  that  compete  for  their  attention”  (para.  4). 
In  this  process,  the  operator  is  supposed  to  decide  which  task  to  attend  to  before 
all  the  other  tasks,  a  phenomenon  based  on  allocation  of  attention. 

Based  upon  these  incident  reports,  in  Figure  2  the  authors  express  the 
significance  of  the  number  of  incidents  in  advanced  cockpit  aircrafts.  One  thing 
important  to  note  here  is  the  decrease  in  the  number  of  incidents  through  the 
years.  Funk,  Suroteguh,  Wilson,  and  Lyall  (1998)  also  suggested  that  it  could  be 
concluded  that  over  time  pilots  became  more  used  to  automation. 
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Figure  2.  Reported  task  prioritization  incidents  between  given  time  periods 
(From:  Funk,  Suroteguh,  Wilson,  and  Lyall,  1998,  p.  4) 
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In  Figure  3,  Funk,  Suroteguh,  Wilson,  and  Lyall  (1998)  summarized  the 
factors  that  could  be  important  in  task  management  performance.  The  numbers 
under  the  references  block  indicate  the  number  of  studies  in  the  literature  that 
has  suggested  the  related  line  as  an  important  factor  in  task  management 
performance.  The  ones  marked  with  “*”  on  the  other  hand,  are  the  suggested 
factors  by  the  researchers,  which  were  not  previously  identified  in  the  literature. 


Factors 

References 

advance  knowledge  of  upcommg 
tasks 

[10] 

discnmmability  of  task-related 
stimuli 

[10] 

differences  in  level  of  effort  required 
to  process  task-related  stimuli 

[10] 

temporal  proximity  of  task-related 
stimuli 

[10] 

task  unportance:  aviate  >  navigate  > 
communication  >  manage  systems 

[13] 

perceived  urgency  of  task  (tune 
remaming  vs.  tune  to  complete) 

* 

task  difficulty 

age 

(automation)  task  proficiency 

* 

task  recency 

* 

task  momentum:  tendency  to 
continue  to  perform  the  current  task 

[13] 

task  proximity  to  completion 

* 

amount  of  effort  already  invested  in 
tasks 

* 

perceived  task  status  (satisfactory, 
unsatisfactory) 

* 

Figure  3.  Factors  affecting  task  prioritization  (From:  Funk  et  al.,  1998,  p.  6) 


lani  and  Wickens  (2004)  found  that  task  switching  time  was  directly 
related  to  salience  of  the  cue.  They  suggested  that:  “people  tend  to  be  more 
proactive  in  task  management  when  workload  is  low  and  more  reactive  when 
workload  becomes  high”  (p.  2). 
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lani  and  Wickens  (2004)  stated  that  even  if  the  task  prioritization  order 
should  follow  ANCS  hierarchy  in  an  effort  to  allocate  the  attention  ideally,  this  is 
not  always  the  pattern  that  performers  follow.  They  noted  there  were  accidents 
because  of  devoting  attention  to  one  task  while  a  higher  priority  task  was 
neglected,  lani  and  Wickens  also  referred  to  an  accident  as  an  example  of  this 
kind  of  neglect,  when  the  pilots  of  an  airliner  were  distracted  by  a  landing  gear 
indicator  failure.  Pilots  devoted  their  attention  to  this  malfunction,  resulting  in  the 
neglect  of  aircraft  control,  and  failure  to  recognize  the  descent  of  aircraft.  Thus, 
the  airliner  crashed  with  the  death  of  all  on  board. 

Freed  (2000)  focused  on  the  task  prioritization  computation  of  agents,  and 
suggested  a  heuristic  prioritization,  when  all  other  factors  were  equal,  with  the 
following  order: 

First  is  urgency,  which  is  the  time  period  until  the  deadline  of  the  task, 
suggesting  completion  of  the  nearer  deadline  first.  Second  is  importance,  which 
depicts  the  consequences  of  missing  the  deadline  of  a  task,  suggesting 
completion  of  the  most  important  deadline  first. 

The  third  factor — duration — is  important  for  two  reasons.  First,  the  task 
duration  might  affect  the  completion  of  other  tasks  before  their  deadline,  and 
second,  depending  on  the  duration  of  tasks  and  given  deadlines,  the  task 
prioritization  order  would  change,  therefore  suggesting  completion  of  the  briefer 
task  first. 

The  last  place  in  this  heuristic  goes  to  interruption  cost,  which  depicts  the 
attention  switching  costs  from  one  task  to  another,  suggesting  completing  an 
ongoing  activity  rather  than  switching  to  another  (Freed,  2000). 

lani  and  Wickens  (2004)  gave  the  factors  affecting  task  prioritization  in  the 
following  order:  task  complexity,  attentional  tunneling,  task  importance,  and 
physical  salience.  They  state  that  switching  attention  to  new  tasks  depends  on 
the  urgency  of  the  ongoing  task,  and  the  attentional  resources  it  allocates.  While 
there  is  an  attentionally  demanding  ongoing  task,  there  will  be  fewer  attentional 
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resources  to  attend  to  the  other  tasks.  At  this  point,  attentional  tunneling  comes 
on  the  table,  because  it  is  not  always  the  case  that  the  operators  use  their 
attentional  resources  on  the  more  complex  tasks,  instead  attending  to  less 
complex  but  more  urgent  ones.  Therefore,  lani  and  Wickens  concluded  that  the 
urgency  of  the  ongoing  task,  not  its  complexity,  is  the  most  significant  factor 
when  it  comes  to  switching  attention  between  concurrent  tasks.  Independent 
from  urgency  and  complexity  there  is  task  importance.  Operators  should 
compare  the  importance  of  tasks  while  allocating  their  attentional  resources, 
suggest  the  authors.  However,  they  suggest  the  physical  salience  of  the  task 
here  to  explain  the  fact  that  tasks  are  not  always  initiated  due  to  their  importance. 
They  pointed  out  evidence  that  memorial  or  less  salient  reminders  are  less  likely 
to  initiate  an  attentional  switch  than  the  salient  reminders.  They  also  pointed  to 
evidence  that  auditory  salience  is  more  effective  in  switching  attention  to  new 
tasks  than  is  visual  salience,  and  the  switching  time  of  higher  priority  interrupting 
tasks  was  much  faster  than  the  lower  priority  ones.  Their  findings  support  this 
evidence,  and  expand  it  by  stating  that  auditory  modality  does  not  deprive  high 
priority  flight  tasks  from  attentional  resources  because  it  works  parallel  to  the 
visual  flight  parameters. 

On  the  other  hand,  Damos  and  Tabachnick  (2001)  found  that  task 
duration  and  structure  had  significance  on  interruption  of  the  ongoing  tasks.  It 
was  seen  that  ongoing  tasks  that  cannot  be  divided  into  sub-tasks  and  are 
relatively  shorter  in  duration  were  not  likely  to  be  interrupted.  They  also  stated 
that  something  which  occurs  frequently  would  indicate  less  time  and  safety 
critical  information,  thus  would  be  most  likely  to  be  assessed  as  too  low  priority  a 
task  to  interrupt  the  ongoing  one. 

Prospective  memory  was  explained  by  Berg  (2002)  as  the  memory  where 
the  intentions  to  take  certain  acts  are  kept  for  the  future.  It  is  expressed  that 
acting  upon  remembering  is  significant,  rather  than  just  remembering  those 
intentions. 


31 


In  a  recent  study,  Dismukes  (2006)  analyzed  aviation  accidents  where 
crew  error  was  the  major  component  of  the  reason.  He  found  out  that  neglecting 
the  regular  operational  sequences  was  a  significant  factor  of  crew  related  errors. 
To  find  out  the  underlying  reasons  for  such  errors  from  experienced  pilots  who 
have  done  the  same  operational  procedures  countless  times,  he  ran  a  study  and 
argued  that  prospective  memory  along  with  multi-task  management  constituted  a 
reason  for  human  error. 

Dismukes  (2006)  characterized  prospective  memory  by  the  following 
traits:  the  intention  to  take  an  action  in  the  future  when  conditions  permit,  time 
between  intention  and  its  performance  (which  is  allocated  with  other  tasks),  and 
the  absence  of  an  indication  to  remind  it  to  the  performer  explicitly.  He  stated  that 
the  crucial  issue  is  to  retrieve  these  intentions  from  the  memory  when  the 
circumstances  permit,  but  not  to  keep  them  in  the  memory.  He  suggested  that 
prospective  memory  plays  a  significant  role  in  the  following  task  situations  in  the 
cockpit:  episodic  tasks,  habitual  tasks,  atypical  actions  substituted  for  habitual 
actions,  interrupted  tasks,  and  interleaving  tasks  and  monitoring.  He  stated  that, 
unlike  habitual  tasks,  episodic  tasks  have  to  be  remembered  to  be  performed. 
Dismukes  added  that  in  the  information  rich  environment  of  cockpits,  the  pilot’s 
attention  might  be  allocated  to  some  other  cues  rather  than  the  ones  that  would 
help  to  retrieve  episodic  tasks.  Habitual  tasks,  on  the  other  hand,  hold  a  risk  of 
omitting  some  important  steps,  especially  when  the  sequence  of  operations  is 
changed  for  some  reason.  Another  phenomenon  related  to  habitual  tasks  is 
when  the  pilots  need  to  substitute  their  habitual  procedures  under  some 
particular  circumstances.  Dismukes  stated  that  pilots  might  attend  to  their 
habitual  procedures  rather  than  the  substituted  one.  He  pointed  out  a  common 
error  where  pilots  do  not  return  from  their  interrupted  tasks  after  accomplishing 
the  interrupting  task.  Findings  of  his  study  showed  that  the  reason  for  this 
behavior  was  not  having  explicit  cues  in  the  perceptually  rich  environment  of  the 
cockpit.  Another  reason  was  the  oncoming  task  demands  right  after  the  end  of 
the  interrupting  task,  which  prevented  the  participants  from  comprehending  the 
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situation  and  retrieving  their  prior  intention  from  the  memory  to  return  to  the 
interrupted  task.  Another  task  demand  that  might  cause  prospective  memory 
retrieval  issues  is  when  the  pilot  needs  to  shift  attention  to  monitor  another  task, 
while  performing  an  ongoing  task.  Dismukes  (2006)  stated  that  even  if  their 
consequences  might  be  significant,  monitoring  low  probability  events  might  be 
difficult  because  a  human  has  a  tendency  to  allocate  his  attention  to  the  sources 
where  he  can  get  more  information. 

Hancock,  Williams  and  Manning  (1995)  stated  that  human  performance  is 
not  a  linear  phenomenon  that  works  directly  proportional  to  the  given  task 
demand.  The  authors  mentioned  that  automation  has  changed  the  pilot’s  role 
from  operator  to  system  manager.  They  referred  to  literature  about  two  contrary 
opinions  about  the  task  demand,  workload  and  human  performance  relationship. 
The  first  opinion  claimed  that  workload  is  directly  related  to  the  task  demand 
characteristics  under  circumstances  where  attention  is  directed  to  the  source  for 
prolonged  periods.  On  the  other  hand,  it  was  claimed  that  workload  and  task 
demand  are  separate  phenomena,  and  under  some  circumstances,  workload 
and  human  performance  are  directly  proportional  to  another,  where  human 
performance  increase  is  observed  as  workload  increased. 

Considering  this  background,  Hancock  and  his  colleagues  made  a  study 
to  point  out  the  relationship  of  task  demand,  workload  and  performance.  Their 
findings  indicated  that  the  perceived  workload  of  a  performer  is  related  to  the 
level  of  task  demand  he  was  subject  to  until  that  time  (e.g.,  pilots  who 
transitioned  from  a  relatively  higher  task-intensive  platform  perceived  the 
workload  as  low,  while  pilots  transitioning  from  low  task-intensive  cockpits 
perceived  the  given  workload  as  high).  Therefore,  the  authors  suggested  that  to 
better  assess  the  current  workload  level,  task/mission  history  has  to  be  taken  into 
account.  This  perception  of  relative  workload  leads  to  the  conclusion  that  pilots 
coming  from  high  to  low  task-intensive  environments  should  perform  better  and 
vice  versa.  The  initial  reflex  is  to  associate  high  workload  with  poor  performance, 
while  linking  low  workload  and  good  performance.  However,  the  authors 

33 


observed  that  transitioning  from  a  higher  task-intensive  cockpit  to  a  lower  task¬ 
intensive  one  did  not  produce  higher  human  performance,  but  lower.  The  authors 
suggested  that  workload  analyses  provide  a  window  of  efficiency  which  may 
enable  proactive  behaviors  in  cockpits  by  potentially  providing  information  before 
the  fact  happens.  They  suspected  that  the  reason  for  uncertainty  in  correlation  is 
nonlinear  human  characteristic,  because  of  some  observations  where 
performance  increased  while  the  tasks  became  harder.  They  commented  on  the 
promising  findings  where  performance  and  workload  appear  to  be  associated  in 
monitoring  tasks,  which  constitute  an  increasing  percentage  of  the  tasks  to  be 
done  in  each  new  cockpit  design. 

In  a  study  to  examine  mental  workload  and  situational  awareness, 
Alexander  and  Nygren  (2000)  stated  that  mental  workload  is  important  in 
assessing  systems,  but  that  when  assessing  the  quality  of  information  that  the 
operator  is  using,  situational  awareness  should  be  also  considered.  In  their 
experiment,  Alexander  and  Nygren  compared  two  different  cockpit  interfaces:  a 
conventional  one,  and  a  virtually  augmented  cockpit  with  advanced  displays. 
They  found  that  mental  workload  was  lower  in  the  virtually  augmented  cockpit 
than  in  the  conventional  one. 

Alexander  and  Nygren  also  stated  that  having  high  situational  awareness 
enables  pilots  to  function  more  effectively  and  with  more  time  awareness.  They 
mentioned  that  even  if  situational  awareness  and  mental  workload  are  inter¬ 
related,  they  are  considered  to  be  independent  elements,  referring  to  the  findings 
where  mental  workload  ratings  increased  with  the  increase  in  task  demands 
while  situational  awareness  ratings  did  not  change.  However,  their  findings 
indicated  a  relationship  between  mental  workload  and  situational  awareness. 
They  observed  that  changes  in  the  experimental  set  up  which  caused  mental 
workload  increase  resulted  in  a  decrease  in  situational  awareness. 

Wilson  (2002)  examined  pilot  workload  using  measures  such  as  heart 

rate,  eye  blinking  rate  and  the  electroencephalographs  (EEG)  of  pilots.  He  stated 

that  increasing  heart  rate  would  indicate  increasing  mental  workload,  and  that 
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decreasing  blink  rate  would  indicate  increasing  visual  demands.  He  noted  that 
these  psycho-physiological  measures  changed  more  than  the  other  phases  of 
flight  at  takeoff  and  landing  phases.  He  also  showed  that  when  pilots  flew  similar 
maneuvers  to  the  ones  they  did  before,  their  workload  ratings  were  lower  than 
when  they  conducted  less  familiar  maneuvers. 

4.  Auditory  and  Visual  Resource  Considerations 

In  a  current  study,  Lee,  Lee  and  Boyle  (2007)  examined  the  effects  of 
voice  interactions  of  drivers  to  assess  the  effect  on  their  attentional  guidance. 
They  predicted  that  voice  interactions  with  in-vehicle  systems,  while  enabling 
them  to  keep  their  visual  focus  on  the  road,  would  cause  additional  cognitive 
load.  Therefore,  this  load  would  cause  delays  in  their  responses  to  regular  events 
such  as  braking  and  showing  reaction  to  traffic  lights,  when  responding  verbally 
to  auditory  messages  rather  than  just  listening  to  them.  In  their  experiment,  the 
main  task  of  their  participants  was  to  follow  a  frequently  braking  lead  car  at  a 
certain  distance.  Concurrently,  the  participants  were  supposed  to  follow  and 
remember  signs,  pedestrians  and  various  similar  targets  in  the  scene,  and  listen 
to  an  auditory  message,  then  respond  to  questions  about  this  message.  This 
experimental  design  composed  a  complex  dynamic  situation  containing  many 
tasks  required  to  be  done  more  or  less  concurrently. 

The  findings  of  the  study  supported  their  predictions.  Lee  and  her 
colleagues  found  that  drivers  showed  slower  reactions  under  dual-tasking 
conditions,  where  both  tasks  required  a  response  from  driver.  This  finding, 
however,  was  mostly  the  result  of  attentional  distraction,  because  the  drivers  had 
to  respond  to  asked  questions,  but  not  because  of  the  additional  information 
given  by  the  auditory  message. 

Similar  to  this  finding,  Damos  and  Tabachnick  (2001)  reported  that  when 
the  ongoing  task  in  the  cockpit  was  auditory,  pilots  reacted  slower  to  the 
interrupting  tasks. 
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The  findings  of  lani  and  Wickens  (2004)  established  the  basis  for  these 
suggestions,  where  they  stated  that  the  auditory  modality,  in  fact,  is  a  factor  that 
supports  the  parallel  processing  of  visual  control  sources. 

According  to  Wickens  and  Ververs  (1998)  the  main  reason  for  putting 
HUDs  in  cockpits  is  to  present  necessary  information  to  the  pilot  on  one  source, 
so  that  he  can  save  his  attentional  sources  for  higher  priority  tasks  rather  than 
using  them  to  re-accommodate  while  switching  from  one  display  to  another  to 
extract  the  necessary  information.  Thus,  it  is  critical  to  understand  the  way 
attention  is  modulated. 

Wickens  and  Ververs  (1998)  pointed  out  evidence  that  a  HUD  creates 
attentional  narrowing,  especially  under  high  workload  situations,  thus  avoiding 
the  assessment  of  presented  data  on  the  HUD  by  the  pilot.  They  referred  to  a 
military  report  to  indicate  the  possible  threats  of  attentional  narrowing  caused  by 
HUDs.  In  this  incident,  the  pilot  failed  to  notice  the  barrier  on  the  runway  due  to 
overloaded  symbology  on  the  HUD,  in  addition  to  the  excessive  brightness, 
resulting  in  an  accident. 

Wickens  et  al.  (2004)  suggested  that  one  significant  issue  is  “to  evaluate 
the  general  tradeoff  between  the  scanning  costs  of  a  separated  display,  and  the 
clutter  costs  of  a  more  integrated  display”  (p.  5).  While  the  clutter  costs  for  target 
detection  have  been  observed  before,  they  found  evidence  that  symbology 
clutter  even  degrades  the  detection  rate  of  expected  targets.  However,  in  some 
cases,  the  benefits  of  some  sources  of  clutter  were  great  enough  to  dominate 
their  costs,  because  they  reduced  the  workload. 

The  data  in  Wickens  and  colleagues’  study  revealed  that  pilots  use  the 
synthetic  display  to  extract  the  attitude  information  even  if  they  can  see  the 
horizon  in  IMC.  This  attentional  preference  establishes  a  significant  threat  to 
pilots,  especially  when  there  is  information  outside  which  is  not  presented  on  the 
synthetic  display.  This  finding  supports  the  automation  complacency  risk 
suggested  by  Wickens  (2000). 
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Wickens  and  Ververs  (1998)  referred  to  basic  attention  and  aviation 
literature  to  point  out  the  findings  that  show  the  negative  effect  on  attention  of 
having  unnecessary  symbology  and  visual  data.  They  state  that  it  is  highly 
significant  to  determine  which  data  is  required  within  the  context  of  any  given 
flight  phase,  following  which  the  rest  should  be  cleared  accordingly.  Their  results 
indicated  that  high  clutter  disrupts  the  target  detection  performance  both  in  near 
and  far  domains. 

The  presence  of  more  symbology  than  is  required  for  the  current  task  was 
pointed  out  to  be  a  factor  making  the  attentional  switching  harder  between  near 
and  far  domains,  due  to  increased  clutter  on  the  display,  thus  resulting  in 
attentional  capture  or  tunneling. 

Yeh,  Wickens  and  Seagull  (1999)  pointed  out  that  the  scanning  cost  of 
head-up  display  clutter  is  less  than  the  scanning  costs  of  head-down  instruments. 
However,  they  also  stated  that  the  cost  of  scanning  of  head-up  displays 
increases  significantly  with  the  increasing  symbology  on  these  displays  intended 
to  present  more  information  to  the  pilot.  They  claimed  that  previous  research  has 
proved  that  pilots  are  more  vulnerable  to  miss  the  information  in  the  real  world 
and  make  errors  with  a  cluttered  display  in  their  field  of  view  because  of  impaired 
vision  due  to  increased  symbology. 

Wickens  and  Ververs  (1998)  analyzed  the  scanning  vs.  clutter  costs,  and 
their  influence  on  near  and  far  domain  target  detection.  They  reported  that  the 
reduction  in  the  costs  of  attention  switching  between  the  head-down  displays  and 
the  scanning  cost  outweigh  the  clutter  cost  of  HUD  in  detecting  environmental 
targets.  HUD  decreased  the  detection  time  of  far  domain  targets  significantly.  On 
the  other  hand,  clutter  adversely  affected  the  near  domain  target  detection 
performance  when  it  came  to  recognizing  the  change  on  the  HUD  symbology,  as 
well  as  far  domain. 

Prinzel  and  Risser  (2004)  commented  that  being  attentionally  captured 
because  of  the  salience  of  the  near  domain  cues,  pilots  experience  difficulty  in 
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switching  their  attention  between  domains;  thus,  they  cannot  process  two 
sources  of  information  at  the  same  time.  They  pointed  out  pilot  reports,  where 
several  of  them  admitted  that  they  sometimes  find  themselves  being  so  fixated 
on  near  domain  cues  that  they  are  totally  unaware  of  anything  else. 

Wickens  and  Ververs  (1998)  found  that  pilots  prefer  to  use  the 
environmental  sources  to  extract  attitude  information,  which  would  explain  why 
they  do  not  keep  their  eyes  down  on  the  flight  instruments  as  much  as  they 
might.  These  results  favor  the  use  of  HUD  over  head-down  instruments,  of 
course. 

Yeh  et  al.  (1999)  found  out  that  cuing  symbology  on  HMD  was  useful  for 
expected  targets  and  helped  subjects  to  point  out  them  out,  while  distracting 
them  from  unexpected  ones.  They  pointed  to  evidence  from  an  air-to-ground 
mission  experiment  indicating  that  cuing  symbology  resulted  in  erroneous 
decisions,  making  pilots  target  non-target  locations. 

This  result  is  compatible  with  the  habits  of  pilots  using  HUD.  In  an  air-to- 
ground  attack,  pilots  continuously  track  the  approximate  target  location  while 
pulling  the  aircraft  to  that  direction  to  put  the  target  in  the  field  of  view  of  HUD. 
When  the  HUD  field  of  view  covers  the  area,  the  cuing  symbology  is  seen 
overlaying  on  the  target.  This  order  enables  pilots  to  avoid  computer  generated 
error,  as  well  as  human  error,  because  it  gives  opportunity  to  double  check  and 
assess  whether  the  correct  target  has  been  acquired.  Therefore,  it  is  also  fair  to 
claim  that  pilots  using  HMD  are  more  susceptible  to  targeting  errors  unless  their 
systems  work  perfectly,  because  the  continuous  symbology  in  the  field  of  view  is 
going  to  discourage  them  from  focusing  their  attention  on  the  far  domain. 

Yeh  et  al.  (1999)  stated  that  when  there  is  a  contradiction  between  the 
information  acquired  from  the  real  world  and  the  information  from  computer 
generated  symbology  (automation);  the  pilots’  decisions  were  to  trust  in  the 
automation-based  information.  Their  results  support  this  claim  by  stating  that  the 
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presence  of  cuing  information  might  draw  attention  to  a  certain  area,  thus 
withdrawing  attention  from  surrounding  areas. 

Yeh  and  colleagues  also  suggested  that  cuing  symbology  on  HMD’s  could 
increase  attentional  tunneling,  after  their  experiment  about  using  an  HMD  and 
computer-generated  fault  imagery  for  aircraft  inspection.  They  also  stated  that 
“the  presence  of  cuing  may  result  in  an  inappropriate  allocation  of  attention:  an 
overreliance  on  an  automation-based  cue”  (p.  539-540).  Further,  they  suggested 
that  as  the  amount  of  information  presented  in  the  field  of  view  of  the  pilot 
increases,  the  target  detection  rate  decreases  because  of  the  allocation  of  the 
pilot’s  attention  to  the  near  domain. 

Johnson,  Wiegmann  and  Wickens  (2005)  found  that  as  the  attention 
allocated  to  a  certain  area  of  interest  increases,  the  task  performance  of  the 
pilots,  which  require  other  areas  of  interest,  decreased  proportionally  because 
the  pilots  could  not  allocate  the  necessary  amount  of  attention  to  these  sources. 

Johnson,  Wiegmann  and  Wickens  also  stated  that  the  performance  of 
pilots  on  certain  tasks  that  are  based  on  visual  scanning  is  also  negatively 
affected  by  hi-tech  cockpit  displays,  as  is  the  visual  scanning  itself.  They 
suggested  that  if  pilots’  visual  scanning  is  interrupted  by  the  in-cockpit  displays, 
thus  drawing  attention  into  the  cockpit  and  away  from  the  world  outside,  their 
assessment  on  weather  conditions  will  become  weaker  over  time. 

Wickens  (2002)  indicated  that  salient  display  cues  are  beneficial  to  remind 
pilots  of  the  tasks  to  be  done,  thus  mentioning  the  importance  of  allocation  of 
attention  to  better  task  management.  Connected  to  this  statement,  Wickens 
criticized  the  task  management  strategies  that  direct  the  pilot’s  attention  to  a 
certain  direction  to  inform  him  about  the  flight  data.  He  stressed  the  importance 
of  training  in  task  management  by  defining  what  makes  a  pilot  a  better  task 
manager,  and  stated  that  the  key  to  becoming  a  better  pilot  is  the  ability  to  share 
the  attentional  sources  between  the  outside  world  and  the  informative  cockpit 
displays. 
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Bohnen  and  De  Reus  (2004)  stated  that  one  of  the  most  significant 
indicators  of  mental  workload  is  the  visual  allocation  of  attention,  and  used  it  as  a 
manipulator  to  find  out  the  effect  of  number  of  displays  on  pilot  workload.  Their 
findings  indicated  that  as  the  number  of  displays  associated  with  concurrent 
tasks  increased,  pilots  could  not  maintain  the  same  flight  performance  because 
of  limited  attentional  sources.  However,  they  also  found  that  increasing  the 
number  of  displays  did  not  bring  additional  mental  load,  thus  pointing  out  the 
visual  allocation  of  attention  as  the  reason  for  performance  decrease.  On  the 
other  hand,  they  observed  that  pilots  developed  new  strategies  to  manage 
attentional  sources  and  maximize  flight  performance  while  doing  the  other 
concurrent  tasks.  Thus,  the  authors  expressed  the  importance  of  considering 
pilots’  attentional  management  strategies  next  to  visual  allocation  of  attention 
while  predicting  the  workload. 

Horrey,  Wickens  and  Consalus  (2006)  stated  that  as  more  technologies 
are  implemented  in  cars,  a  safety  concern  arises  because  these  new 
implementations  are  going  to  compete  for  the  limited  visual  and  attentional 
resources  of  drivers,  thus  reducing  the  resources  allocated  for  the  primary  task, 
driving.  They  claimed  that  the  eye  is  kind  of  a  “single-server  queue”  and  visual 
scanning  is  the  server  of  this  queue. 

Horrey  et  al.  (2006)  expressed  the  importance  of  expectance  and 
information  bandwidth  by  stating  that  performers  become  more  likely  to  monitor 
the  displays/information  sources  where  they  find  relevant  information  more 
frequently  than  others.  They  showed  evidence  of  this  phenomenon  in  that  the 
experienced  drivers  were  observed  to  have  more  extensive  visual  scanning  while 
novice  drivers  became  fixated  on  cars  around  them  during  a  task  where  they  had 
to  track  their  lane  in  a  demanding  road  environment.  This  observation,  the 
authors  suggested,  indicates  that  experienced  drivers  have  a  better  idea  of 
where  to  expect  and  extract  the  relevant  information  than  do  novice  ones. 
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Horrey  and  his  colleagues  suggested  that  a  scanning  pattern  should 
optimize  the  cost-benefit  effectiveness,  where  it  increases  the  benefits  of 
scanning  while  decreasing  the  costs  of  missing  information. 

In  an  experiment,  Horrey  and  his  colleagues  (2006)  used  the  SEEV 
(Salience/Effort/Expectancy/Valuable)  model  where  they  intended  to  point  out  the 
influence  of  in-vehicle  tasks  on  driving  performance  and  visual  scanning  patterns. 
They  explained  the  parameters  of  this  model  as  follows:  salience  is  the 
obviousness  of  information  on  an  information  source,  effort  is  the  visual  angle 
difference  between  informational  sources,  expectancy  is  a  combination  of 
bandwidth  and  value  of  the  information  on  a  display  which  indicates  the  task 
relevance  and  importance  of  information  for  the  given  task,  and  value  is  the 
importance  of  that  information  compared  to  the  others.  The  authors  pointed  out  a 
weakness  of  the  SEEV  model  by  stating  it  is  built  on  the  “momentary  allocation  of 
focal  (foveal)  vision”  while  there  are  various  tasks  where  necessary  information 
can  be  extracted  through  ambient  vision  (p.  68).  This  evidence  indicates  that 
performers  do  not  necessarily  need  to  fixate  their  attention  on  focal  vision  for  all 
the  information  sources.  Focal  vision  is  said  to  be  linked  directly  to  eye 
movements,  and  used  for  visual  search  and  tasks  such  as  reading,  which  require 
“high  visual  acuity,”  while  ambient  vision  is  mostly  used  for  spatial  orientation, 
and  postural  control  in  locomotion. 

Prinzel  and  Rlsser  (2004)  stated  that  a  large  part  of  the  world  in  humans’ 
vision  is  continuously  monitored  by  ambient  vision,  and  only  a  smaller  part  by 
foveal  vision.  They  stated  that  these  two  vision  systems  work  in  a  parallel 
fashion,  without  competing  for  the  same  attentional  source. 

Horrey  et  al.  (2006)  showed  evidence  that  experienced  drivers  can  use 
their  ambient  vision  more  effectively  than  novice  ones  for  lane  tracking,  even 
without  moving  their  eyes  to  direct  their  attention  on  the  road. 

The  findings  of  Horrey  and  his  colleagues  showed  an  increase  in 
performance  proportional  to  the  increase  in  allocation  of  focal  visual  sources  that 
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was  related  to  the  increase  in  value  of  that  particular  task.  Therefore,  it  has  been 
seen  that  as  the  priority  of  that  task  increases,  more  attention  is  directed  to  that 
task,  which  results  in  better  performance.  Their  findings  also  indicated  that  as  the 
complexity  of  in-vehicle  systems  and  their  bandwidth  increases,  the  lane-keeping 
performance  decreases  due  to  switching  attention  from  the  road  to  in-vehicle 
displays.  Drivers’  response  time  to  safety  critical  situations,  which  is  dictated  by 
the  time  for  the  eyes  to  move  from  down  inside  the  vehicle  to  the  road,  also 
increased,  pointing  out  the  role  of  focal  vision.  However,  the  correlation  between 
focal  vision  and  hazard  reaction  times  is  not  that  strong,  suggesting  that  ambient 
vision  is  also  used  for  hazard  detection  and  reaction,  though  still  requiring  focal 
vision  to  correctly  identify  and  react.  This  study  examined  only  one-task 
conditions.  No  multi-task  performance  was  examined,  so  the  increase  in  task 
performance  when  the  allocation  of  attention  to  one  source  might  not  prove  to  be 
beneficial  under  high  workload  conditions.  However,  the  findings  are  significant 
to  support  the  importance  of  task  prioritization  in  allocating  the  attentional 
resources  to  the  required  task. 

Horrey  and  his  colleagues’  (2006)  results  showed  that  increasing  the 
frequency  or  complexity  of  in-vehicle  systems  influenced  the  driving  performance 
more  than  the  manipulation  of  outside  factors,  indicating  the  importance  of 
allocation  of  focal  vision  attention. 

5.  Decision  Making  Considerations 

Decision-making  often  becomes  very  demanding  in  the  cockpits.  In  order 
to  mitigate  the  problems,  it  is  crucial  to  understand  the  decision-making  process, 
and  better  decision-making  practices,  via  the  literature. 

Doherty  (2001)  mentioned  Klein’s  “recognition-primed  decision  making 
model  (RPD)”  (p.  15).  The  model  says  that  experienced  pilots  relate  their  current 
situations  to  their  stored  situations  from  their  experiences  and  make  correct 
decisions  easily.  Considering  the  capabilities  of  JSF,  this  kind  of  decision-making 
ability  will  be  crucial.  In  high-risk,  limited  time  situations  and  with  lots  of  available 
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information  via  the  displays,  it  will  be  very  important  to  make  the  correct 
decisions  quickly.  As  Doherty  mentions,  the  required  experience  for  RPD  can  be 
built  in  various  ways  such  as  real  or  training  missions  and  training  systems. 

Similarly,  Spencer  (2000)  discussed  that  in  many  situations,  the  experts 
make  quick  decisions  depending  on  their  experience.  Over  time  and  training, 
they  build  a  repository  for  many  situations,  and  if  a  critical  situation  requires  a 
very  quick  reaction,  they  retrieve  the  closest  match  and  decide  their  next  move. 

The  multirole  capability,  increased  number  of  complicated  sensors,  and 
sophisticated  autopilot  are  the  main  features  of  JSF  that  create  differences  from 
previous  fighters.  If  one  also  considers  the  missions  that  JSF  will  perform  in  the 
future,  it  becomes  obvious  that  the  pilots  will  be  under  high  workloads  in  limited 
time  frames  while  performing  their  tasks.  Thus,  properly  trained  automated 
responses,  like  RPD,  will  become  more  crucial. 

A  typical  mission  scenario  example  will  be  very  explanatory  in  terms  of 
mission  nature:  during  composite  air  operations  (COMAO),  where  multiple  Air-to- 
Air,  Air-to-Ground,  friendly  and  hostile  assets  are  in  the  battle  area,  it  will  be 
highly  likely  that  a  JSF  pilot  will  be  looking  at  a  complicated  picture  with  a  lot  of 
information  in  the  cockpit.  Considering  the  time  constraints,  and  other  stressors, 
it  is  easy  to  conclude  that  the  pilot’s  ability  to  look  at  the  displays,  filter  the  data, 
and  evaluate  it  for  a  quick  and  correct  decision  will  be  very  challenging.  And  from 
the  beginning  of  their  exposure  to  JSF,  the  pilots  will  have  access  to  all  of  these 
sources  of  information.  Considering  the  number  and  the  complexity  of  the 
systems,  a  considerable  amount  of  training  will  be  needed  to  achieve  the  ability 
of  accomplishing  the  aforementioned  tasks. 

6.  Models  to  Predict  Allocation  of  Attention 

Even  if  the  prediction  of  attention  allocation  is  beyond  the  scope  of  the 
current  study,  the  authors  of  this  thesis  believe  that  briefly  mentioning  some 
models  can  help  to  understand  how  the  process  of  human  attention  is  captured. 
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Fleetwood  and  Byrne  (2004)  analyzed  three  models  of  visual  attention 
allocation  to  point  out  the  attentional  patterns  and  where  the  performer  is  going  to 
direct  his  attention  next.  They  focused  on  the  following  factors  to  determine  to 
what  extent  they  influence  the  attentional  patterns  of  the  performers:  the 
frequency  of  information  generation  on  the  monitored  display,  the  probability  of 
critical  information  coming  up  while  monitoring  another  display,  the  cost  of 
missing/detecting  critical  information,  and  the  cost  of  monitoring. 

The  first  model  they  analyzed  was  the  Senders  model.  This  model 
concentrates  on  the  frequency  of  information  generation  (this  factor  also  is  called 
the  bandwidth  of  related  instrument/display)  and  the  alarm  frequency  of  this 
instrument.  However,  Fleetwood  and  Byrne  expected  this  model  to  predict  less 
accurately  when  under  complex  task  conditions  because  only  the  bandwidth  can 
be  manipulated.  This  expectation  might  indicate  that  this  model  is  less  likely  to  be 
used  in  aviation,  where  the  complexity  increases  due  to  higher  workload. 

The  second  model  Fleetwood  and  Byrne  examined  is  the  SEEV  model. 
The  name  of  the  model  stands  for  Salience,  Expectancy,  Expectancy  and  Value, 
which  constitute  the  factors  that  this  model  considers  to  influence  the  allocation 
of  attention.  Authors  expect  a  high  and  relatively  accurate  prediction  rate  from 
this  model,  unless  its  results  are  extrapolated  for  more  specific  ones.  Fleetwood 
and  Byrne  stated  that  the  information  extracted  from  a  source,  the  effort  to 
monitor  that  source  and  the  effort  to  switch  attention  from  one  information  source 
to  another  are  the  factors  considered  by  the  Information  Foraging  model.  They 
said  that  SEEV  and  Information  Foraging  models  predict  that  the  sampling 
frequency  of  an  information  source  is  relevant  to  the  value  of  the  information 
extracted  from  that  source. 

7.  FAA  Research  about  Flight  Training 

The  main  consequences  of  modern  aircraft  are  the  enhanced  capabilities, 
increased  number  of  sensors,  and  automation.  A  very  important  and  useful  set  of 
research  related  to  the  human  factors  concerns  in  modern  cockpits  is  the 
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research  conducted  by  the  FAA  about  general  aviation.  The  research  began 
approximately  in  2003  with  a  final  report  analyzing  the  mishaps  of  modern 
general  aviation  aircraft,  and  continued  to  evaluate  almost  to  the  present  day. 
Another  important  aspect  of  the  research  is  the  solutions  proposed  about  the 
training. 

The  first  related  study  is  the  final  report  by  Fiduccia  et  al.  (2003).  Their 
study  is  based  on  1 1  accidents,  all  of  which  had  issues  related  to 
“Technologically  Advanced  Aircraft”  (TAA).  Their  definition  of  TAA  is: 

a.  IFR-certified  GPS  navigation  equipment  (navigator)  with  moving 

map;  or 

b.  A  multi-function  display  (MFD)  with  weather,  traffic  or  terrain 

graphics;  and 

c.  An  integrated  autopilot,  (p.  9) 

This  definition  has  similar  parts  to  modern  fighter  jets,  in  terms  of  the 
MFDs,  GPS  and  autopilot.  It  is  legitimate  to  see  their  study  as  a  case  study  on 
the  accidents  by  Subject  Matter  Experts.  And  as  an  important  conclusion, 
Fiduccia  and  his  colleagues  claim  that  the  traditional  flight  training  did  not 
address  the  use  of  relatively  modern  technological  systems,  and  that  a  study  on 
how  to  address  all  required  issues  of  the  new  technology  needed  to  be 
conducted.  This  study  established  the  basis  for  the  following  related  studies  that 
sought  solutions  to  this  common  training  concern. 

French,  Blickensderfer,  Ayers  and  Connolly  (2005)  conducted  one  of  the 

follow-on  studies  in  order  to  study  the  differences  between  the  “maneuvers 

based  training  (MBT)”  and  the  “scenario-based  training  (SBT)”  (p.  3).  The  MBT  is 

generally  the  approach  of  the  traditional  instrument  training  used  so  far,  and  the 

SBT  was  the  proposed  solution  by  the  TAA  research  team  after  conducting 

research  on  the  accidents  in  TAAs.  They  explain  that  SBT  is  a  student  centered 

training  approach  in  which  the  students  declare  their  problematic  areas  and  build 

appropriate  and  realistic  training  scenarios  to  improve  them.  After  this  process,  it 

is  the  instructor’s  responsibility  to  come  up  with  appropriate  “performance 
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measures.”  Then,  the  instructor  decides  the  success  of  the  trainee  using  those 
performance  measures  and  gives  feedback.  The  main  point  of  SBT  is  that  “...the 
scenario  is  the  curriculum”  (p.  6).  French  and  colleagues  also  explain  the 
traditional  training  approach  as  based  on  MBT.  The  repetition  of  the  required 
maneuvers  makes  the  trainees  proficient.  The  maneuvers  are  not  implemented 
to  scenarios,  and  the  student  only  follows  the  syllabus,  but  this  does  not 
contribute  to  the  flow  of  his  training  with  deciding  what  to  practice  in  which 
realistic  scenario. 

French  and  colleagues  conducted  their  study  with  27  participants  who 
were  not  experienced  in  TAA.  Those  participants  ran  through  a  pretest,  received 
8  hours  of  training  with  either  MBT  or  SBT,  and  then  completed  another  test 
thereafter.  They  divided  the  phases  of  flight  into  eight  categories  and  graded 
them  separately  with  a  “blind  rater.”  Additionally,  they  also  used  subjective 
surveys  in  order  to  learn  personal  opinions  and  get  deeper  insight.  Their  results 
revealed  that  the  trainees  with  SBT  received  higher  grades  than  MBT  trainees  in 
five  out  of  eight  categories  with  a  statistical  significance,  and  in  the  remaining 
categories  their  grades  were  equal.  The  authors  also  add:  “Further,  the  SBT 
group  demonstrated  a  tendency  to  report  reduced  workload  and  an  improvement 
in  self  efficacy  and  situational  awareness  compared  to  MBT”  (French  et  al. ,  2005, 
p.  3). 

Robertson,  Petros,  Schumacher,  McHorse  and  Ulrich  conducted  one  of 
the  FAA  sponsored  studies  in  2006.  The  purpose  of  the  study  was  to  measure 
the  differences  between  the  “problem-based  learning”  studied  by  the  FITS  and 
the  traditional  training  approaches.  In  PBL,  the  scenarios  should  be  prepared 
such  that  they  force  the  trainees  to  face  more  challenging  and  harder  cognitive 
processes,  and  to  evaluate  and  compare  more  than  one  alternative  during  the 
decision  making  process.  They  reported  that  FAA  has  accepted  the 
shortcomings  of  the  current  training  approaches  in  “atypical”  conditions. 
Regardless  of  whether  a  situation  was  normal  or  abnormal,  it  was  designated  as 
typical  when  the  standard  response  to  that  situation  was  covered  in  the  training. 
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They  defined  the  “atypical”  situations  or  hard  conditions  as  those  with  many 
alternative  solutions  that  were  complex,  and  which  were  not  covered  during 
standard  training  (Robertson,  Petros,  Schumacher,  McHorse,  &  Ulrich,  2006,  p. 
2). 

To  teach  the  required  skills  to  pilots  for  giving  them  the  ability  to  make  the 
proper  decisions  during  the  atypical  conditions,  it  is  necessary  to  address  the 
complex  or  advanced  decision  making  skills  during  the  training.  And  they 
reported  that  the  current  training  systems  do  not  address  this  issue;  that  was  one 
of  the  objectives  of  their  study  to  evaluate  “Higher  Order  Thinking  Skills”  in  their 
training  system  (Robertson  et  al. ,  2006,  p.  3). 

Robertson  and  colleagues  conducted  an  experiment  using  a  simulation 
program  in  a  personal  computer.  The  three  groups  were  used  for  a  FITS 
preferred  PBL  approach,  or  the  traditional  training  approach  and  self  study  group. 
Each  group  was  given  a  test  before  they  began  their  training  with  a  TAA  aircraft 
simulation,  went  through  their  training  and  at  the  end  took  a  posttest  of  their  new 
TAA  aircraft.  Their  participants  were  college  students  who  had  private  pilot 
license  and  were  certified  to  fly  in  Instrument  Flight  Conditions.  The  traditional 
and  self-study  groups  were  given  traditional  training  during  the  ground  lessons, 
briefings  and  flight  lessons.  They  were  asked  multiple  choice  or  straightforward 
questions  requiring  direct  answers.  The  PBL  group  was  given  questions  via  the 
scenarios  that  required  the  students  to  do  further  research  with  their  documents. 
Also,  the  answers  to  these  scenario-based  questions  were  not  as  straightforward 
as  the  other  groups’  questions;  rather,  they  necessitated  more  intensive  mental 
efforts  (Robertson  et  al.,  2006). 

Both  subjective  and  objective  measurements  were  used.  The  simulation 

game  had  the  feature  of  data  collection  and  saving.  That  feature  was  used  to  get 

the  objective  measurements  about  the  parameters  of  interest  such  as  altitude, 

airspeed,  etc.  SA  was  measured  by  the  questions  asked  by  the  researchers, 

while  they  stopped  the  simulation.  “Aeronautical  Decision  Making”  was  measured 

subjectively  by  the  observing  researchers,  and  objectively  by  examinations  about 
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the  HOTS.  They  reported  that  the  participants  with  PBL  performed  significantly 
better  under  complex  conditions,  and  added  “The  findings  also  reflected 
improvements  in  the  indicators  of  aeronautical  decision-making  (pilot  judgment) 
and  a  reduction  in  the  number  of  mistakes  made  by  the  pilot”  (Robertson  et  al., 

2006,  p.  60). 

Dornan,  Beckman,  Gossett  and  Craig  (2007)  are  the  last  researchers  of 
the  FAA  research.  One  of  the  recommendations  of  their  report  was  adding 
“consequences”  to  scenario-based  trainings.  They  claimed  that  without  any 
consequence,  the  scenario-based  training  would  not  be  fully  useful,  because  to 
come  up  with  a  solution  will  require  less  effort.  But  when  adding  serious 
consequences  to  the  scenario,  the  trainees  will  be  forced  to  process  more 
information,  and  use  more  cognitive  resources;  thus,  the  end  result  of  the  training 
will  be  more  beneficial.  The  example  they  had  for  such  an  input  was  injecting  a 
“...transplant  organ  to  the  destination  airport”  in  a  possible  divert  decision 
scenario.  Apparently  after  going  through  that  kind  of  hard,  challenging  decision, 
the  trainees  will  gain  more  than  without  any  consequence  injected  training  (p.  3). 

Another  method  of  exploring  the  proper  decision-making  skills  is  looking  at 
the  experts.  Many  studies  have  been  conducted  for  that  purpose,  and  point  out 
similar  phenomenon.  Spencer  (2000)  discussed  this  common  finding  that  in 
many  situations,  the  experts  make  quick  decisions  depending  on  their 
experience.  Over  time  and  training,  they  build  a  repository  for  many  situations, 
and  if  a  critical  situation  requires  a  very  quick  reaction,  they  retrieve  the  closest 
match  and  decide  their  next  move. 

The  cockpit  automation,  load  on  visual  resource,  informational  overload, 
cognitive  resources,  attentional  allocation,  training  considerations  and  many 
other  topics  covered  in  this  chapter  provide  a  background  on  all  of  the  required 
aspects  of  human  factors  concerns  in  modern  glass  cockpit  aircraft.  The 
following  chapters  benefit  from  the  relevant  studies  mentioned  in  this  chapter, 
both  in  problem  identification  and  training  recommendation  purposes. 
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C.  THE  IMPORTANT  FEATURES  OF  JSF  RELEVANT  TO  HUMAN 

FACTORS 

One  of  the  important  objectives  of  this  thesis  lies  in  understanding  the 
features  and  capabilities  of  JSF.  The  authors  believe  that  JSF  has  many  new 
capabilities  and  systems  leading  to  new  required  skills  and  human  factors 
concerns.  In  this  chapter,  these  topics  as  well  as  the  differences  among  fighter 
cockpits  are  provided.  First,  the  features  and  major  systems  of  JSF  are 
introduced  from  unclassified  documents,  and  then  possible  human  factors 
concerns  are  predicted  from  the  literature  and  the  operational  experience  of  the 
authors. 

1.  General  Features  and  Systems  of  JSF 

JSF  is  intended  to  be  a  "...  multi-branch,  multinational,  supersonic  fighter,” 
which  is  planned  to  replace  most  of  the  current  generations  of  aircraft  in  the 
contributing  countries’  air  force  inventories.  With  its  advanced  features  and  three 
variations  for  different  platforms,  it  offers  multi-role  capabilities.  Fusing  the  data 
from  many  sensors  onboard  the  aircraft  is  claimed  to  be  the  major  strength  of  its 
design  (Jensen,  2005,  para.  2).  Even  if  the  flight  performance,  maneuverability 
and  G-performance  are  not  necessarily  superior  to  the  previous  generations,  the 
systems  presented  with  this  aircraft  offer  new  capabilities  and  operational 
concepts.  Next  to  the  improvements  on  already  existing  systems,  some  highly 
intuitive  systems  are  introduced  to  the  aviation  community  for  the  first  time,  such 
as  the  electro-optical  distributed  aperture  system  (DAS).  Even  if  there  are  many 
other  points  to  consider  assessing  JSF,  its  capabilities,  and  pilot  vehicle 
interface,  the  authors  think  the  following  systems  are  more  likely  to  affect  mission 
effectiveness. 

The  most  striking  difference  and  innovation  of  the  JSF  cockpit  is  the 
8”x20”  liquid  crystal  display  screen  right  in  the  middle  of  the  forward  panel 
(Figure  4).  Almost  all  of  the  information  from  the  sensors  and  systems  are 
presented  to  the  pilot  via  this  display,  which  can  be  modified  by  touch  screen 
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buttons  to  reach  and  modify  any  desired  system  data.  This  display  will  be  the 
major  PVI  system  between  the  pilot  and  aircraft. 


Figure  4.  JSF  cockpit  with  LCD  display  suite  (From:  briefing  received  from  PVI 

Team  at  Lockheed  Martin,  Fort  Worth,  TX) 


The  forest  of  toggle  switches  in  previous  fighter  cockpits  has  been 
wiped  clean  from  the  F-35’s  interior  landscape,  with  most  of  their 
functions  moved  to  the  touch  screen.  A  few  switches  still  sprout 
here  and  there,  but  the  overall  cockpit  ambience  is  one  of 
simplicity  and  calm,  almost  to  the  point  of  aeronautical  feng  shui. 

(Kent,  2006,  para.  3) 

These  features  give  the  impression  that  it  will  be  very  intuitive  and  easy  to 
operate  the  systems  of  JSF. 

After  providing  a  general  picture  about  the  capabilities  and  cockpit  design 
of  the  JSF  above,  it  will  be  necessary  to  focus  on  the  individual  systems.  The 
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following  systems  and  their  features  do  not  cover  all  of  the  systems  and 
capabilities  of  JSF,  but  are  thought  to  be  the  important  ones  relevant  to  the 
scope  of  this  thesis  (related  to  human  factors  concerns).  Limited  information  can 
be  found  online  and  in  some  published  resources.  But  because  JSF  is  still  under 
development,  there  is  much  restricted  information  due  to  security  concerns,  and 
the  following  information  is  cited  directly  from  unclassified  sources  online.  For 
further  information,  the  readers  can  follow  the  references. 

2.  Radar 

The  AN/APG-81  fire  control  radar,  developed  by  Northrop  Grumman,  has 
the  following  general  features: 

•  Terrain  mapping  with  high  resolution,  able  to  cover  an  area  three  to  four 
times  wider  than  existing  radars.  This  feature  enables  pilots  to  have  better 
assessment  of  the  area  of  interest  by  giving  high  resolution  and  wider  field 
of  view  (Jensen,  2005).  This  feature  might  also  introduce  attentional 
tunneling  issues  along  with  itself  because  pilots  might  fixate  their  attention 
on  this  display  to  search  and  acquire  the  target. 

•  Provides  both  air-to-air  and  air-to-ground  target  information  at  the  same 
time.  This  is  a  significant  capability  to  provide  the  pilot  a  better  situational 
awareness  in  a  multi-threat  environment,  where  he  can  assess  all  the 
given  parameters  and  make  quicker  and  more  accurate  decisions 
(Jensen,  2005).  This  capability  is  not  present  in  current  generation 
fighters.  However,  it  may  present  a  challenge  to  informational 
management,  where  the  pilot  should  be  able  to  filter  the  unnecessary  data 
and  focus  on  the  required  ones.  Pilots,  apparently,  are  required  to  manage 
their  mental  resources  more  effectively  in  order  to  cope  with  ample  data 
and  make  better  decisions. 

•  It  can  be  slewed  to  any  other  sensor’s  field  of  view,  whether  the  other 
sensor  is  onboard  or  off  board  (Jensen,  2005).  This  feature  amplifies  the 


51 


situational  awareness  and  the  coordination  among  friendly  assets; 
however,  given  information  does  not  allow  any  further  assessment  in 
terms  of  human  factors  concerns. 

3.  Electro-optical  Targeting  System  (EOTS) 

Along  with  the  AN/APG-81  radar,  the  EOTS  is  one  of  the  major  sensors  of 

JSF. 

•  It  consists  of  non-active  sensors  operating  both  during  night  and  day; 
enemies  are  not  able  to  intercept  any  signals  because  it  doesn’t  radiate 
any  signal. 

•  It  provides  very  detailed  IR  images  to  the  pilots,  and  is  said  to  be  an 
additional  way  of  acquiring  and  analyzing  targets  in  addition  to  the  radar. 
The  pilots  will  be  able  to  acquire  the  targets  beyond  visual  ranges,  and 
sync  the  radar  with  EOTS,  in  order  to  examine  the  targets  in  more  detail. 

•  For  Air-to-Air  purposes,  it  has  an  Infrared  Search  and  Tracking  System 
(IRSTS),  and  for  Air-to-Ground  purposes  a  Forward  Looking  Infrared 
Radar  (FUR).  Any  off-board  system  can  target  laser  to  target,  and  those 
lasers  can  be  locked  by  a  tracker  in  JSF  (Jensen,  2005). 

Apparently,  the  pilots  will  focus  on  both  radar  and  EOTS  in  many  task 
situations.  Focus  and  attentional  distribution  as  well  as  having  robust  mental 
models  on  both  systems  will  be  crucial  with  such  advanced  systems. 

4.  Distributed  Aperture  System  (DAS) 

The  electro-optical  distributed  aperture  system  (DAS)  has  the  following 
features: 

•  Consists  of  six  IR  cameras. 

•  Provides  vision  through  the  body  of  the  aircraft. 

•  Works  in  collaboration  with  the  helmet  mounted  display  (HMD)  to  relay  a 

continuous  360  degrees  of  passive  environmental  information  to  pilot. 
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•  Gives  the  pilot  “missile  approach  warning,  countermeasures  deployment, 
passive  air-to-air  radar,  off-axis  targeting  for  air-to-air  missiles,  and  wide 
field-of-view  day/night  pilot  vision”  (Jensen,  2005,  para.  20) 

•  The  integrated  data  from  all  cameras  can  be  superimposed  over  a  tactical 
display  which  can  be  reached  via  data  link  from  ground  units  or  another 
flying  asset  (Jensen,  2005). 

The  DAS  intends  to  increase  the  situational  awareness  of  the  pilot  while 
not  overloading  him  with  excess  information.  The  system  itself,  along  with  its 
ability  to  work  in  collaboration  with  data  linked  tactical  pictures,  is  intended  to 
ease  the  pilot’s  workload  of  adapting  himself  to  the  tactical  arena, 
comprehending,  deciding  and  taking  action  (Jensen,  2005). 

However,  being  such  a  new  system,  these  authors  think  that  it  will  take 
some  time  and  operational  experience  to  be  thoroughly  assessed  by  means  of 
flight  safety  in  certain  conditions.  Previous  studies  done  on  visual  attention 
allocation  are  helpful  to  consider  the  possible  attentional  tunneling  issues 
associated  with  this  system  and  its  presentation  on  HMD.  However,  future 
research  should  be  encouraged  by  air  forces  operating  this  aircraft,  not  only 
depending  on  simulator  experience  but  also  pilot  feedback  from  real  operational 
missions  in  various  weather  and  mission  scenarios.  A  question  that  arises  is  pilot 
disorientation.  What  kinds  of  effects  will  the  DAS  cause  to  pilots  after  “looking 
through”  the  fuselage  and  returning  back  to  normal  vision  in  bad  weather 
conditions?  These  types  of  concerns  should  be  considered  during  training  to 
cope  with  any  possible  problems.  Another  issue  is  the  balance  between  the 
capabilities  of  the  aircraft  and  the  pilots.  Apparently,  the  aircraft  has  the  potential 
to  dominate  the  pilot  with  ample  data  and  many  other  capabilities.  But  how  to 
raise  pilots’  capabilities  to  a  point  that  they  can  fully  benefit  from  the  data  safely 
and  efficiently  without  fixating  or  becoming  disoriented  seems  to  be  a  challenge 
during  training. 
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5. 


Communication/Navigation/Identification  (CNI)  and  Other 
Capabilities 


The  CNI  system  will  be  approximately  similar  to  the  most  modern  current 
generations,  but  the  officials  report  that  it  will  be  more  tailored  to  fit  in  “network 
centric  warfare”  (Jensen,  2005).  The  system  is  also  reported  to  provide  such 
functions  as  beyond-visual-range  identification  friend  or  foe,  secure  voice 
communications,  caution  and  warning,  intercom,  and  intraflight  information 
sharing  among  multiple  aircraft  via  high-speed  broadband  data  link”  (Jensen, 
2005,  para.  29).  JSF  will  also  have  other  current  data  link  capabilities  to  share 
information  (Jensen,  2005).  The  aforementioned  capabilities  will  take  JSF  one 
step  further  in  terms  of  information  dominance,  and  common  tactical  picture; 
thus,  ability  to  filter  and  evaluate  the  data,  and  make  proper  decisions,  will 
probably  be  an  important  training  requirement  with  all  of  the  on-  and  off-board 
data  sources.  The  design  of  JSF  will  probably  achieve  the  point  that  the  pilots  are 
no  longer  challenged  to  collect  information,  but  to  use  that  vast  amount  of  data. 

Another  innovative  application  in  the  cockpit  is  reported  to  be  the  3-D 
sound  system.  “Three-dimensional  audio  algorithms,  to  direct  appropriate  audible 
cues  360  degrees  around  the  pilot  are  expected  to  be  part  of  the  CNI  suite's 
future  growth”  (Jensen,  2005,  para.  33).  This  feature  will  apparently  reduce  the 
visual  workload  in  some  cases,  and  increase  SA  easily  during  high  workload 
situations.  Distribution  of  signals  or  information  to  visual  or  auditory  displays  (or 
both),  establishing  proper  standard  operational  procedures,  and  training  for  pilots 
will  be  necessary  to  enable  them  to  benefit  from  this  new  capability. 

D.  POTENTIAL  RESEARCH  CONCERNS  ABOUT  HUMAN  FACTORS  IN 

JSF 

Based  on  the  research  literature  review  and  considering  the  features  of 
JSF,  the  following  questions  are  the  major  areas  of  research  this  thesis  tries  to 
identify: 
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Concern  1.  What  are  the  potential  areas  that  could  cause  Negative 
Transfer  of  Training  (NTT)  problems  during  transition  period? 

Concern  2.  How  will  the  pilots  filter  the  data  and  focus  only  to  the  required 

ones? 

Concern  3.  What  kind  of  new  skills  will  be  needed  for  JSF? 

Concern  4.  What  kind  of  concerns  can  pose  problems  about  pilot-autopilot 
interaction  in  JSF? 

Concern  5.  Which  systems  of  JSF  are  going  to  cause  problems  to  the 
pilots,  and  thus  need  to  be  addressed  carefully  during  training? 
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III.  METHODOLOGY 


A.  INITIAL  SURVEY  AND  INTERVIEWS 

The  main  objective  of  this  thesis  is  to  identify  and  point  out  the  critical 
components  in  transition,  to  follow  the  phases  of  JSF  in  terms  of  Human  Factors 
considerations,  and  then  to  discuss  the  possible  ways  to  avoid  any  problems 
beforehand. 

The  initial  reflex  was  to  focus  on  the  accidents  and  mishaps  that  were 
suffered  during  and  after  the  previous  transition  phases  of  any  type  of  aircraft, 
and  then  to  start  building  up  from  them.  By  this  way,  the  authors  planned  to 
figure  out  the  common  transition  concerns,  design  a  flight  scenario  that  would 
cover  the  most  possible  human  errors  along  with  a  task  analysis  of  that  particular 
mission  scenario,  and  then  run  a  simulator  or  part-task  trainer-based  experiment 
to  observe  the  human  error  tendencies  of  pilots  from  various  flight  backgrounds. 
These  steps  could  provide  to  the  authors  enough  bases  to  analyze  the  common 
errors  and  finally  offer  training  guides  considering  the  different  aircraft  types 
before  JSF. 

Throughout  the  thesis  research,  the  authors  realized  that  there  is  no 
specific  transition  study  available  in  public  sources  that  would  be  directly  useful 
for  the  thesis,  but  only  negative  transfer  of  training  studies,  which  is  a  dominating 
factor  in  transition  accidents.  While  brainstorming  about  how  to  put  these  ideas 
together  most  efficiently,  the  authors  had  a  chance  to  make  a  field  trip  to 
Lockheed  Martin  in  Fort  Worth,  Texas.  The  preliminary  survey  held  during  this 
trip  of  the  pilots  experiencing  the  JSF  mission  simulator  has  totally  changed  the 
insight  about  this  study,  which  will  be  explained  in  detail  later  on. 

The  authors  think  that  explaining  the  reasons  and  background  of  this 
change  itself  will  provide  a  broad  perspective  about  the  major  Human  Factors 
considerations  and  shifts  of  the  required  skills  in  JSF. 
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This  chapter  mentions  the  initial  thesis  plan;  discusses  the  field  trip  to  the 
Lockheed  Martin  facilities  in  Fort  Worth  for  preliminary  surveys,  interviews,  and 
personal  experience  in  an  unclassified  simulator;  and  explains  the  modified  plan 
with  the  rationale  behind  it. 

1.  Initial  Plan 

The  initial  plan  of  this  thesis  was  to  seek  NTT  issues  that  may  be 
encountered  during  the  transition  period.  The  main  human  factors  concern  was 
considered  to  be  problems  during  transition  period,  and  no  other  areas  were 
thought  to  pose  problems  afterward.  But  the  trip  to  Lockheed  Martin  changed  the 
strategy,  and  the  authors  felt  impelled  revise  the  thesis  strategy.  This  chapter 
explains  the  process  of  this  revision  throughout  this  research. 

a.  Negative  Transfer  of  Training 

Accidents  expedite  investigation  and  research  on  causes  and 
solutions.  Accidents  also  motivate  pilots  in  training  to  understand  the 
circumstances  and  to  avoid  them  in  the  future. 

Repetition  is  one  of  the  key  components  of  learning.  If  a  particular 

procedure  or  action  is  followed  regularly  for  a  period,  it  turns  into  an  automated 

response  and  the  human  brain  takes  care  of  that  procedure  without  the  need  to 

pay  attention  to  that  process.  This  phenomenon  can  be  explained  by  the 

following  example.  In  the  first  few  weeks  of  driving  of  a  car,  one  needs  to  check 

the  locations  of  the  controls  of  some  systems,  such  as  air  conditioning,  turn 

signals,  windows,  etc.  After  some  time,  the  operator  gets  used  to  their  locations 

and  can  control  them  even  without  double-checking.  This  is  because  our  brain 

matches  that  location  with  that  particular  system,  and  one  does  not  need  to  direct 

his  attention  especially  to  these  small  operational  issues.  He  just  sends  signals  to 

his  brain  that  he  wants  to  switch  the  lanes  and  take  the  left  one,  his  body  takes 

the  necessary  action  automatically  and  operates  the  signal  handle  even  without 

thinking.  Now  imagine  that  he  changes  his  car,  and  it  has  a  different  user 

interface  in  which  the  signal  handle  works  the  opposite  way.  In  the  beginning, 
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when  he  wants  to  switch  lanes,  he  will  find  himself  giving  the  signal  to  wrong 
direction,  thus  causing  some  confusion  for  him,  and  probably  unsafe  situations 
for  the  traffic  flow.  This  “negative  transfer  of  training  (NTT)”  is  a  common 
phenomenon.  Boldovici  (1987)  explains  it  as  “Practicing  Task  A  interferes  with 
learning  or  performing  Task  B”  (p.  239).  When  imagining  this  scenario  on  the 
ground,  it  does  not  sound  that  critical.  However,  when  things  require  a  quicker 
response,  this  kind  of  confusion  might  pose  a  life  threatening  safety  problem. 
This  is  why  pilots  are  prone  to  safety  problems  in  transition  to  a  new  type  of 
aircraft  more  than  at  any  other  time.  Experiences  and  automated  reflexes  are 
hard  to  unlearn  and  replace  with  new  ones.  Aviation  is  a  very  demanding  task 
both  physically  and  mentally.  Therefore,  pilots  are  required  to  follow  certain 
procedures,  usually  in  very  short  time  periods.  This  poses  a  very  insidious 
danger  for  pilots,  especially  for  those  who  have  gained  strong  habits  after  many 
flight  hours  in  another  type.  As  pilots  acquired  many  automated  skills,  this 
becomes  a  concern  especially  during  transition  phases.  Apparently,  it  is  hard  to 
learn  the  new  operational  procedures  that  conflict  with  experience. 

This  phenomenon  formed  the  basis  of  the  initial  thesis  plan:  to 
identify  and  point  out  the  possible  areas  prone  to  NTT.  The  major  expectation 
was  to  identify  switches  operating  in  an  adverse  way  or  critical  switches  in  a 
different  location,  and  those  operational  or  display  concepts  conflicting  with 
experiences  depending  on  particular  flight  experience.  For  instance,  the  so-called 
“Jettison”  switch  in  JSF  could  be  in  the  same  location  as  the  “Master  Arm”  switch 
of  a  previous  type.  This  could  lead  to  problems  such  as  an  inadvertent  jettison  of 
the  stores  while  trying  to  arm  them.  Another  instance  could  be  symbology 
conventions  that  operate  in  a  different  way  to  indicate  the  state  of  the  aircraft  or 
some  avionic  systems. 

b.  Task  Analysis 

Besides  identifying  the  switches  or  displays  that  might  cause 
problems,  the  authors  also  considered  the  operational  procedures  required  in 
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typical  mission  scenarios.  Air  Force  officials  provided  some  secret  documents 
about  JSF  that  included  how  to  operate  the  avionics  and  other  Pilot  Vehicle 
Interfaces.  Those  documents  are  meant  to  be  used  for  a  task  analysis  in  order  to 
capture  the  differences  between  the  JSF  and  current  types.  The  next  step  of  the 
task  analysis  plan  was  to  identify  the  potentially  problematic  mission  types  that 
may  pose  greater  risk  to  pilots,  and  then  conduct  the  analysis  on  those  missions. 
The  opinions  from  the  field  trip  and  the  information  provided  by  the  documents 
changed  this  plan.  The  major  reason  preventing  the  task  analysis  was  the 
documents;  they  were  more  like  technical  manuals  than  operating  procedures 
with  checklist  items.  Thus,  they  could  not  provide  enough  background  to  support 
a  robust  task  analysis. 

c.  Experiment 

An  experiment  was  an  important  part  of  the  initial  plan  to  support 
the  thesis  with  statistical  results.  The  authors  planned  to  focus  on  some  research 
questions  after  evaluating  the  preliminary  survey  results,  and  analyzing  pilot 
interface  differences  of  JSF  and  previous  types.  The  next  step  was  to  devise  an 
experiment  scenario,  probably  similar  to  the  one  used  in  task  analysis,  which 
could  enable  the  authors  to  cover  most  of  the  possible  issues.  The  steps  above 
would  give  the  basis  for  experimental  questions  and  once  the  access  to  either 
PC  trainer  or  unclassified  simulator  was  given,  experiments  could  be  conducted 
for  those  questions. 

d.  Security  Classification  and  Required  Literature 

The  JSF  project  is  currently  engaging  in  flight  and  systems  tests, 
and  even  though  there  is  no  major  change  presumed,  there  are  still  some 
developments  especially  in  the  Pilot  Vehicle  Interface  (PVI)  of  the  aircraft.  This 
fact  brings  commercial  security  issues  along  with  military  security  ones.  All  the 
documents  about  JSF  require  a  very  high  level  of  security  clearance.  As  the 
authors  of  this  study,  our  advisors  and  we  had  this  clearance;  however,  this  is  a 
major  issue  for  the  post-thesis  as  well  as  experimentation  period.  Considering 
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the  entities  that  could  make  use  of  or  evaluate  this  study,  security  would  be  a 
major  problem  and  most  of  them  would  not  even  be  able  to  read  it.  So,  at  some 
point  a  compromise  had  to  be  made  in  either  the  scope  of  study,  or  the  benefits 
that  could  be  gained  from  it.  Another  issue  would  be  giving  the  participants’ 
access  to  the  classified  documents  or  devices  during  the  experiment. 

e.  The  Problems  Experienced 

The  first  problem,  as  mentioned  before,  was  the  scarcity  of  the 
literature  on  studying  the  transition  periods.  The  reason  could  be  the  security 
issues  of  the  military.  In  addition  to  the  lack  of  a  similar  transition  study,  there 
were  also  relatively  few  studies  on  negative  transfer  of  training  in  aviation  in 
general.  Available  resources  also  included  U.S.  military  reports,  yet  there  were 
problems  in  finding  appropriate  studies  for  this  thesis. 

The  second  and  most  significant  problem  was  the  security  issue. 
Even  if  the  authors  could  experience  the  unclassified  Pilot  Vehicle  Interface  (PVI) 
unit,  the  security  measures  prevented  them  from  using  any  training  or 
demonstration  device  of  JSF  for  any  further  research  and  experiment  efforts  on 
this  equipment.  Therefore,  an  experiment  devised  by  the  authors  would  not  be 
highly  related  to  the  PVI  of  JSF. 

In  order  to  conduct  a  task  analysis  robustly,  one  has  to  have 
enough  resources  relevant  to  the  operational  procedures  and  enough  insight  to 
the  context.  In  this  case,  the  authors  have  flight  experience  with  the  F-16  and  its 
operational  procedures.  Even  if  they  did  not  have  any  flight  experience  with  JSF, 
access  to  the  operational  procedures  of  the  chosen  mission  scenarios  could 
provide  acceptable  task  analysis  on  JSF. 

Typical  checklists  provide  sequential  procedures  about  how  to 
operate  systems  on  board,  as  well  as  emergency  procedures  (e.g.,  “Battery  - 
ON,  Check  -  Battery  ON  light”),  and  these  procedures  are  to  be  followed  during 
the  related  tasks.  However,  this  information  alone  is  not  enough  for  a  task 
analysis,  because  checklist  information  covers  how  to  operate  systems  and  the 
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actions  to  be  taken  in  case  of  problems.  It  does  not  cover  the  interaction 
between  the  pilot  and  switches  or  displays.  The  document  known  as  “-1” 
(Operating  Manual)  among  aviators  is  needed  for  this  purpose.  The  documents 
provided  in  this  case  were  more  technical  documents  than  an  operational 
manual.  They  had  technical  information  that  would  be  useful  to  explain  how  to 
operate  individual  systems;  however,  there  was  hardly  any  information  to  enable 
one  to  understand  JSF  sequential  operational  procedures.  Conducting  a  task 
analysis  with  these  resources  could  lead  to  wrong  conclusions.  For  these 
reasons,  the  initial  thesis  plan  was  revised,  which  will  be  discussed  in  the 
following  chapters. 

2.  Preliminary  Survey 

Ten  fighter  pilots  were  assigned  to  fly  the  JSF  simulator.  Pilots  had  flight 
experience  at  different  levels,  both  on  flight  hours  and  aircraft  types.  They  were 
assigned  to  fly  various  missions,  both  Air-to-Air  and  Air-to-Ground,  and  they 
employed  various  weapons  accordingly.  They  flew  the  simulator  missions  from 
the  beginning  of  tactical  scenarios,  and  stopped  by  the  end  of  each  scenario. 
They  skipped  ground  procedures,  take-off  and  landing  phases,  and  the 
navigation  phases.  Even  if  these  are  major  phases  to  be  analyzed  for  a  transition 
study,  the  authors  think  that  the  available  mission  phases  were  still  sufficient  to 
support  the  objective  of  this  thesis.  This  gave  a  perfect  opportunity  to  determine 
the  PVI  issues  of  the  JSF  cockpit,  at  first  hand. 

The  opinions  of  the  pilots  were  highly  valuable  for  two  reasons.  First, 
these  pilots  have  the  same  kind  of  flight  training  and  experience  as  those  who 
will  fly  JSF  in  the  very  near  future,  and  some  of  them  probably  will  do  so.  So, 
what  they  experienced  in  the  simulator  will  not  be  much  different  than  what  will 
be  experienced  in  the  future,  and  most  probably  there  will  be  even  more  issues 
due  to  additional  tasks.  Second,  this  was  the  first  JSF  simulator  experience  for 
most  of  the  ten  pilots.  Therefore,  this  was  a  relatively  similar  first  exposure 
scenario  even  if  they  did  not  receive  all  ground  training  that  the  actual  JSF  pilots 
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will.  The  lack  of  ground  training  could  be  beneficial  for  the  study,  because  the 
possible  NTT  issues  would  be  more  apparent.  Another  point  was  that  the 
participants  came  to  the  study  from  flying  experience  with  varying  fighter 
platforms,  so  the  need  was  anticipated  to  figure  out  the  differences  and 
similarities  between  pilots  of  various  types,  and  also  their  transition  suitability  to 
this  new  platform. 

To  validate  whether  the  study’s  initial  plan  was  on  the  right  track,  the 
authors  prepared  a  preliminary  survey  with  open-ended  questions.  The  main 
objective  was  to  have  a  general  understanding  of  problematic  areas  and  to 
validate  the  approach  of  this  study.  The  questions  were  related  to  general  human 
factors  issues  and  did  not  require  any  answers  with  security  classification. 

The  preliminary  survey  questions  were  as  follows: 

1 .  What  is  your  first  impression  about  JSF? 

2.  What  are  the  specific  strengths  and  weaknesses  of  JSF?  (Pilot  cockpit 
interaction) 

3.  What  are  the  similarities/differences  of  JSF  from  your  current  type  of 
aircraft? 

4.  Where  do  you  think  the  potential  areas  are  that  might  be  difficult  for  a 
transition  pilot  from  your  current  type  of  aircraft? 

5.  Generally,  is  the  data  represented  to  the  pilot  at  a  sufficient  level,  or 
did  it  happen  that  you  became  overwhelmed  by  over¬ 
representation/finding  and  filtering  necessary  information? 

6.  Considering  your  current  aircraft  type,  is  there  any  system  in  JSF  that 
serves  the  same  purpose  with  different  operation  principles/data 
representation/interface/interaction? 

7.  Comparing  the  JSF  cockpit  with  your  current  type,  is  there  any 
interface/switch  that  looks  similar,  and/or  is  in  the  same  location,  but 
used  for  another  system/purpose? 

8.  What  are  your  impressions  about  the  specifications  of  HMD?  (Weight, 
dimensions,  Field  of  View,  contrast  and  brightness  range  and  settings) 
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9.  Does  the  HMD  take  your  focus/attention  from  priority  issues  and 
distract  you  from  prioritized  task? 

10.  Do  you  have  any  previous  HUD/HMD  experience?  If  so,  are  there  any 
differences?  (data  representation  locations,  style  and  colors) 

11.  What  are  your  impressions  about  the  usage  and  switching  of  HMD 
modes  and  its  data  representation? 

12.  What  is  your  general  impression  about  HMD?  What  are  its  specific 
strengths  and  weaknesses? 

13.  Do  you  have  any  previous  MFD  experience?  If  so,  are  there  any 
differences?  (data  representation  locations,  style  and  colors) 

14.  What  are  your  impressions  about  the  usage  and  switching  of  MFD 
modes  and  its  data  representation? 

15.  Does  the  increased  symbology  confuse/overwhelm  you? 

16.  What  is  your  general  impression  about  MFD?  What  are  its  specific 
strengths  and  weaknesses? 

17.  What  is  your  impression  about  seat/stick/throttle  positioning  and  their 
usage? 

18.  What  is  your  impression  about  switch  positioning  on  throttle/stick?  Is 
there  any  switch  that  results  in  confusion  or  contradicts  your  previous 
experience? 

19.  What  is  your  impression  about  3D  audio  in  operational  usage? 

20.  Are  there  any  issues  regarding  the  order  sequence  of  operational 
procedures  between  the  JSF  and  your  current  type? 

21.  In  which  type  of  missions/flight  phases,  is  the  workload  of  the  pilot 
increased? 

22.  What  are  the  effects  of  increased  automation  on  your  workload,  SA 
and  flight  concentration? 

23.  What  would  you  like  it  to  be  changed  in  cockpit,  and  how? 

24.  What  do  you  think  a  pilot  in  your  position  would  have  to  learn  and/or 
unlearn  to  fly  JSF? 
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As  they  are  straightforward,  the  questions  only  require  general  human 
factors  and  PVI  issues,  but  not  any  specific  information  about  any  systems.  The 
main  intent  is  to  capture  the  potential  threats  or  problematic  areas  for  various 
aircraft  types  along  with  any  NTT  issues.  The  preliminary  survey  was  handed  to 
the  participants  after  they  had  approximately  six  missions  in  the  simulator,  and 
still  had  at  least  this  much  more  to  fly,  so  that  they  could  focus  on  human  factors 
issues  with  the  consideration  of  survey  questions. 

3.  Interviews 

It  is  always  a  possibility  that  researchers  might  miss  some  important 
points  in  their  survey  questions.  That  was  the  main  reason  it  was  decided  to 
interview  the  participants  in  addition  to  the  preliminary  survey.  The  advantage 
was  that  the  authors  also  are  fighter  pilots  and  have  the  same  language  and 
background  with  participants.  This  helped  a  lot  while  capturing  unmentioned 
points  in  the  survey,  and  enabled  coverage  of  broader  areas. 

Considering  the  flexibility  of  a  mutual  conversation,  and  the  opportunity  to 
interview  all  the  participants,  the  authors  went  over  the  questions  one  by  one. 
Noting  the  process,  they  could  go  deeper  into  the  issues  that  each  participant 
brought  to  the  table.  The  authors  mentioned  their  own  experience  with  the 
unclassified  PVI  device  to  the  participants,  and  discussed  their  experience  on  the 
same  issues.  Since  everyone  had  a  different  approach  to  answering  the  survey 
questions,  the  authors  also  crosschecked  the  answers  given  by  other  participants 
to  validate  answers  as  much  as  possible.  In  conclusion,  the  interviews  supported 
the  findings  from  the  survey,  enabled  the  authors  to  go  deeper  on  many  subjects, 
and  provided  more  valid  results. 

4.  The  Unclassified  Simulator 

The  unclassified  simulator  had  the  full  cockpit  interfaces.  It  was  restricted 
on  any  weapon  employment  procedures,  but  capable  of  all  other  flight  tasks.  The 
authors  flew  the  simulator  approximately  30  minutes  each,  and  performed  take¬ 
off,  basic  navigation,  acrobatic  maneuvers,  operation  of  the  portals  (4  MFD 
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Windows)  on  the  primary  LCD  display,  and  landing.  They  did  not  have  an 
opportunity  to  experience  the  revolutionary  Helmet  Mounted  Display  (HMD)  and 
related  systems  that  work  accordingly  with  it  such  as  Distributed  Aperture 
System. 

B.  FOLLOW-ON  SURVEY 

1.  General  Information 

The  trip  to  Lockheed  Martin  in  Fort  Worth,  TX  changed  the  track  of  this 
thesis  substantially.  Both  the  initial  and  revised  thesis  plans  are  thoroughly 
discussed  in  the  related  chapter,  but  the  major  point  is  that  there  was  a  need  to 
identify  the  human  factors  concerns  further  and  in  more  detail  before  proceeding 
with  the  scientific  process  of  proposing  solutions  and  testing  them.  This  need 
was  the  basis  for  the  follow-on  survey  that  will  be  discussed  in  this  chapter.  As 
the  initial  survey  consisted  of  open-ended  questions,  and  the  interviews  yielded 
important  but  general  results,  the  authors  decided  to  construct  a  survey  based  on 
their  experience  from  the  trip  and  the  literature  in  a  more  quantifiable  way.  The 
following  sections  will  explain  the  follow-on  survey  in  more  detail. 

a.  The  Structure  of  the  Follow-on  Survey 

The  objectives  of  the  following  survey  were  different  than  the  initial 

survey.  The  main  objective  of  the  follow-on  survey  was  to  identify  the  possible 

human  factors  concerns  in  JSF.  The  reason  to  conduct  a  second  survey  was 

twofold.  First,  the  initial  trip  with  the  survey,  interviews  and  self-experience  in 

unclassified  simulator  yielded  different  results  than  the  authors  predicted  the 

human  factors  issues  in  JSF  would  be.  And  secondly,  whatever  results  the  initial 

survey  yielded,  they  were  in  an  open-ended  format.  Many  issues  emerged  from 

the  interviews  and  initial  survey  in  totally  unquantifiable  ways.  Thus,  the 

objectives  of  the  follow-on  survey  were  to  confirm  the  areas  that  emerged 

strongly  in  the  initial  survey,  to  further  investigate  the  areas  that  emerged  weakly, 

to  get  the  opinions  of  the  participants  to  solve  the  possible  problems  and  still 

provide  participants  some  open-ended  questions  related  to  both  problem 
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identification  and  solutions.  One  of  the  important  points  is  the  structure  itself.  The 
initial  survey  consisted  of  totally  open-ended  answers,  whereas  the  follow-on 
survey  benefited  from  a  Likert  scale  in  seven  levels.  This  process  helped  the 
authors  to  understand  the  agreement  levels  of  the  participants  more  accurately 
than  the  initial  survey. 

Another  important  factor  for  the  follow-on  survey  was  to  capture  all 
of  the  areas,  systems,  or  parts  of  the  missions  from  a  human  factors  perspective. 
For  this  reason,  the  survey  is  divided  into  five  segments. 

In  the  first  segment,  the  questions  try  to  capture  and  confirm  the 
important  points  about  controlling  the  aircraft  and  the  possible  use  of  autopilot  in 
JSF.  Whether  or  not  the  basic  manual  flight  skills  will  be  also  important  in  JSF, 
the  usability,  operation  and  expected  usage  of  the  autopilot  are  the  areas  the 
questions  investigate. 

Two  of  the  apparent  changes  or  differences  of  JSF  from  previous 
generations  are  its  modern  and  capable  systems,  and  the  unique  display  suite  for 
pilot  vehicle  interfaces.  And  those  areas  are  questioned  in  the  second  segment 
of  the  survey.  Whether  it  is  hard  to  learn  how  to  operate  the  systems  and  menus, 
the  systems  needing  higher  workloads  that  are  prone  to  cause  disorientation  and 
SA  related  problems,  and  which  are  prone  to  fixation  were  among  the  areas 
investigated  in  this  segment  of  the  survey. 

The  third  segment  is  inspired  by  the  prediction  that  the  JSF  will 
mainly  require  mental  skills  and  cognitive  resources  due  to  high  information  load 
provided  to  pilots  and  its  improved  overall  capabilities  provided  by  new  and 
improved  systems.  The  possibility  of  informational  overload,  data  filtering  issues, 
task  overload  and  workload  concerns,  demand  for  mental  resources  and  abilities 
and  other  attentional  areas  are  investigated  in  this  segment. 

It  is  commonly  accepted  that  the  flight  mission  begins  with  the 
preparations  before  flight,  and  ends  with  the  debriefing.  The  more  capable  an 
airplane  is,  the  more  preparation  time,  briefing  and  debriefing  time  it  needs. 
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Because  there  are  more  capabilities  and  more  sophisticated  systems,  there  must 
be  better  preparation,  more  detailed  briefings  and  further  coordination  among  the 
formation  members.  In  order  to  predict  the  concerns  about  the  mission  planning 
and  briefings,  the  fourth  segment  of  the  survey  contains  questions  about  the 
possible  use  of  simulators  and  desktop  trainers. 

It  is  inevitable  to  miss  some  of  the  factors  during  the  surveys,  and 
that  is  the  reason  for  including  a  segment  with  open-ended  questions.  The  fifth 
segment  of  the  follow-on  survey  attempts  to  learn  pilots’  opinions  about  many 
important  issues  of  JSF  from  a  human  factors  perspective.  The  most  important 
safety  and  training  concerns  the  participants  foresight,  the  prediction  for  the 
transition  phase,  whether  tactical  experience  of  the  pilots  was  important  for  the 
transition  phase,  and  becoming  combat-ready  faster  in  JSF;  these  are  among  the 
areas  the  segment  investigated.  Likert  Style  Survey  items  were  used,  in  which 
participants  indicated  agreement  with  these  statements  from  disagree  to  agree. 

Follow-on  survey  questions  were  constructed  as  follows: 

Question  1:  There  will  be  much  more  use  of  “autopilot”  in  the  JSF 
compared  to  my  current  aircraft  type. 

Question  2:  No  matter  how  good  the  autopilot  is,  pilots  will  still  need 
to  train  basic  flying  skills  as  much  as  previous  type. 

Question  3:  Use  of  the  JSF  autopilot  will  greatly  help  pilots  to  focus 
on  the  tactical  situation. 

Question  4:  The  autopilot  and  other  cockpit  automation  will  result  in 
possible  loss  of  situational  awareness  regarding  the  state  of  the  aircraft  position 
control  and  flight  status. 

Question  5:  The  various  flight  operating  modes  in  the  JSF  are  easy 
to  learn  and  distinguish. 

Question  6:  It  is  easy  to  switch  between  the  modes  of  autopilot  and 
transit  from  autopilot  to  manual  flying. 
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Question  7:  The  pilot  can  easily  capture  any  problems  of  autopilot 
(awareness  of  aviate  &  navigate  the  A/C)  when  accomplishing  other  tactical 
tasks  in  the  cockpit. 

Question  8:  Because  the  autopilot  of  JSF  is  highly  sophisticated 
and  has  various  modes,  basic  flying  skills  are  not  required  as  much  as  it  is 
required  at  previous  types  of  aircrafts. 

Question  9:  Based  on  my  experience  in  the  JSF  simulator,  I  believe 
that  the  flight  management  system  is  easy  to  set  up  and  operate. 

Question  10:  There  are  some  modes  in  the  flight  management 
system  that  I  found  difficult  to  use. 

Question  11:  There  were  instances  that  I  encountered  when  flying 
the  JSF  simulator  for  which  I  did  not  understand  how  to  activate  or  use  the 
appropriate  operating  mode. 

Question  12:  As  far  as  I  can  tell  there  should  be  no  difficulty 
learning  how  to  configure  the  cockpit  displays  for  flying,  navigating  and 
communicating. 

Question  13:  It  may  take  extra  training  time  for  pilots  to  learn  how  to 
effectively  operate  the  new  JSF  cockpit  controls  and  displays. 

Question  14:  There  had  been  instances  that  I  had  to  focus  my 
attention  mostly  to  head-down  displays  to  manage  the  systems  and  reach  the 
information  I  needed. 

Question  15:  Even  if  there  is  a  lot  of  information  from  various 
sensors  on  the  same  display,  I  did  not  have  any  difficulty  to  filter  and  evaluate  the 
data  for  decision-making. 

Question  16:  DAS  can  cause  disorientation  under  some  conditions. 

Question  17:  The  idea  of  seeing  HMD  symbology  wherever  I  look 
did  not  distract  my  attention. 
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Question  18:  I  believe  the  HMD  failure  will  dramatically  affect  the 
mission  efficiency. 

Question  19:  Managing  the  switches  on  throttle  and  stick  effectively 
will  require  a  considerable  amount  of  experience  and  training. 

Question  20:  The  appeal  of  the  head-down  displays  and  the 
workload  need  to  be  done  on  those  displays  might  cause  flight  safety  issues. 

Question  21:  I  believe  that  there  needs  to  be  special  training  to 
teach  pilots  how  to  use  the  expanded  display  suite. 

Question  22:  Without  proper  training  and  experience  pilots  may  not 
be  able  to  handle  the  vast  amount  of  information  provided  by  the  JSF  system. 

Question  23:  I  believe  the  main  task  of  the  pilot  will  switch  from 
mostly  flying  the  aircraft  to  making  tactical  decisions. 

Question  24:  Being  able  to  follow  the  whole  tactical  arena  did  not 
affect  my  focus  on  my  own  target/area  of  interest. 

Question  25:  Managing  both  A/A  and  A/G  data  at  the  same  time  will 
overload  pilots  under  some  tactical  situations. 

Question  26:  I  felt  the  need  to  effectively  filter  and  declutter  the 
presented  information  in  most  tactical  situations. 

Question  27:  Even  if  JSF  presents  a  very  good  tactical  picture,  a 
high  level  of  tactical  experience  is  required  to  be  able  to  use  the  capabilities  of 
the  aircraft  to  the  utmost  extent. 

Question  28:  Compared  to  my  current  type  of  aircraft,  the  training 
period  should  be  longer  to  comprehend  the  systems  thoroughly  and  fly  the 
aircraft  at  its  capabilities. 

Question  29:  The  new  concept  of  JSF  requires  building  and 
maintaining  better  SA  and  more  cognitive  workload  than  my  current  type  of 
aircraft. 
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Question  30:  There  were  some  instances  where  I  had  difficulties  at 
shifting  my  attention  between  the  overall  tactical  picture  and  my  task  related 
tactical  picture. 

Question  31:  I  believe  a  longer  pre-flight  preparation  is  needed  for 

JSF. 


Question  32:  Even  if  the  systems  enhance  in-flight  mutual  support 
at  a  great  level,  formation  briefing  and  coordination  are  even  more  critical  than 
for  previous  types. 


Question  33:  The  simulator  flights  and  real  flights  should  be  exactly 
similar  in  terms  of  briefing,  mission  and  debriefing. 


Question  34:  PC  trainers  donated  with  real  throttle  and  stick 
controls  would  be  significantly  beneficial  to  improve  the  systems  management 
skills  of  pilots. 


Question  35:  To  improve  the  pilots’  display  suite  management  and 
tactical  picture  assessment  skills,  alternative-training  systems  on  the  ground  will 
be  helpful  other  than  actual  flight  conditions. 

Open-ended  questions  are  constructed  as  follows: 


Question  1:  What  do  you  foresee  as  the  most  significant  problems 
or  training  issues?  Briefly  describe. 

Question  2:  Which  one  of  the  following  pilot  types  do  you  think  will 
qualify  to  effectively  and  safely  fly  all  the  missions  with  JSF  earlier  in  transition 
phase:  a  pilot  who  gained  experience  in  another  aircraft  type,  or  a  new  graduate 
pilot  from  flight  school?  Why? 

Question  3:  What  might  be  the  most  likely  cause  for  flight  safety 
problems  in  JSF? 


Question  4:  What  would  be  your  recommendations  about  the 
transition  and  training  phase  of  JSF? 


Question  5:  Other  Comments 
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b.  Participants 

The  participants  of  the  initial  and  follow-on  surveys,  as  well  as  the 
interviews  have  strengths  in  terms  of  scientific  approach,  but  some  other  points 
are  the  drawbacks  that  limit  the  power  of  the  results. 

There  were  same  10  participants  for  both  the  initial  and  follow-on 
surveys  (N=10).  Eight  of  them  are  current  F-16  pilots,  and  two  of  them  are 
current  F-4  pilots.  All  of  them  have  both  Air-to-Air  and  Air-to-Ground  experience, 
and  the  major  positive  side  for  the  participants  is  that  they  were  representative 
for  the  sake  of  the  study.  All  of  them  have  the  common  flight  backgrounds  and 
training  that  the  JSF  pilots  will  have  in  the  Turkish  Air  Force.  They  are  all  fighter 
pilots  from  various  aircraft  types  flown  in  TuAF  and  are  qualified  to  perform  as 
mission  commanders. 

If  the  sample  size  included  many  more  participants  from  various 
types  of  fighter  aircraft,  it  would  be  possible  to  compare  the  results  depending  on 
the  aircraft  origins.  This  could  give  important  clues  for  each  aircraft  type.  It  is 
possible  that  pilots  of  one  aircraft  type  could  think  an  issue  as  a  possible 
problem,  whereas  other  type’s  pilots  wouldn’t. 

The  last  limiting  factor  is  that  the  pilots  were  all  from  the  same 
country.  The  JSF  will  be  flown  in  many  countries,  and  because  the  pilots’  training 
and  abilities  may  vary  depending  on  the  countries,  participants  from  other 
countries  could  provide  a  broader  spectrum  for  human  factors  under 
investigation. 

The  reason  for  not  having  a  larger  sample  size  was  the  security 
and  accessibility  issues  of  the  JSF  project.  JSF  is  under  development  and  has 
commercial  security  issues  as  well  as  military  security  concerns. 

It  is  also  necessary  to  mention  the  experiences  of  the  participants 
with  JSF.  Obviously,  the  actual  pilots  transitioning  to  JSF  will  undergo  very 
demanding  ground  training  phases  before  they  step  into  the  actual  aircraft,  but 
this  was  not  the  case  with  participants.  They  were  provided  with  PVI  documents 
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required  to  operate  the  necessary  systems  in  the  cockpit  and  had  a  couple  of 
weeks  to  study  them.  It  is  legitimate  to  say  that  the  participants  are  not  at  the 
same  experience  and  knowledge  level  that  the  actual  transition  pilots  will  be,  but 
in  contrast  they  were  also  not  experiencing  the  actual,  real  missions.  It  is  to  be 
expected  that  the  real  missions  will  pose  higher  levels  of  risk/stressors,  and  will 
be  much  more  demanding  than  the  simulator  missions  they  flew. 

The  second  important  issue  about  the  participants  on  JSF 
experience  is  the  missions  they  flew  in  the  simulators.  They  flew  the  classified 
mission  simulator  for  both  Air-to-Air  and  Air-to-Ground  missions  for  two  weeks. 
The  positive  side  of  their  experience  can  be  seen  in  that  it  was  their  first 
exposure  to  the  JSF,  as  it  will  be  for  the  first  transition  pilots  in  the  future;  thus, 
their  opinions  are  important  for  providing  first  glance  input.  In  the  second  week, 
they  also  had  little  more  experience  than  a  first  glance  exposure,  and  provided 
helpful  feedback.  But  again  there  is  the  important  factor  of  actual  versus 
simulator  missions.  The  actual  aircraft  will  have  full  cockpit  capabilities  once 
released  to  the  air  forces,  and  the  missions  will  be  higher  risk,  and  much  more 
stressful  and  demanding. 

Even  though  there  were  several  shortcomings  due  to  the  restricted 
sample,  the  authors  believe  that  the  results  are  reliable  enough  for  initial  problem 
identification  and  human  factors  concerns  investigation. 
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IV.  RESULTS  AND  DISCUSSION 


A.  PRELIMINARY  SURVEY  WITH  INTERVIEW  RESULTS 

1.  Findings  from  Interviews,  Preliminary  Survey  and  Own 
Simulator  Experience 

Overall,  the  iterative  answers  to  both  the  preliminary  survey  and  the 
interview  topics  revealed  the  following  outcomes  for  each  question. 

Briefly,  the  survey  and  interview  results  indicated  that  the  most  significant 
issue  will  not  be  the  negative  transfer  of  training,  contrary  to  what  was  expected. 
The  major  issues  about  the  transition  to  JSF  appear  to  be  adaptation  to  new 
technologies  on  combined  flight  displays;  ability  to  evaluate,  comprehend  and 
use  the  vast  amount  of  information  collected  from  various  sensors  covering  all 
the  aircraft;  and  operating  the  automated  systems.  The  findings  showed  that  the 
challenge  for  the  pilots  is  going  to  be  the  increased  mental  workload  compared  to 
previous  aircraft. 


a.  Summary  of  Preliminary  Survey  Findings 

The  following  answers  were  selected  from  the  preliminary  survey 
and  interviews  and  are  the  most  agreed  upon  ones  about  particular  questions,  or 
sometimes  the  interesting  ones.  For  convenience,  the  authors  did  not  write  down 
the  entire  answers  one  by  one,  but  rather  summarized  them  by  common 
explanations  as  much  as  possible. 

1 .  What  is  your  first  impression  about  JSF? 

Almost  all  of  the  participants  expressed  their  first  impression  that 
they  were  not  expecting  such  a  modern  and  capable  fighter.  The  most  significant 
important  input  about  this  question  was  that  the  JSF  cockpit  and  PVI  were  highly 
adaptable.  The  authors  were  expecting  to  get  some  answers  that  could  be 
attributed  to  negative  transfer  of  training,  yet  there  was  no  answer  to  indicate 
that.  One  of  the  strongest  comments  about  the  JSF  cockpit  interface  was  that  it 
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will  require  a  lot  more  mental  workload  and  situational  awareness  to  be  able  to 
use  all  the  information  presented  to  the  pilots.  There  were  many  occasions  where 
the  participants  commented  that  there  might  be  SA  related  accidents,  mishaps 
and  losses  due  to  overwhelming  task  load  and  frustration. 

2.  What  are  the  specific  strengths  and  weaknesses  of  JSF  (Pilot 
cockpit  interaction)? 

The  common  opinion  about  this  question  was  that  the  capabilities 
of  the  aircraft  were  both  its  strengths  and  its  weaknesses.  They  were  the 
strengths  as  they  improve  the  mission  effectiveness  dramatically,  yet 
weaknesses  in  that  they  require  considerable  amounts  of  mental  effort  to  operate 
compared  to  previous  types.  Obtaining  information  is  not  a  challenge  as  before, 
but  filtering,  evaluating  and  making  decisions  while  flying  the  jet  were  seen  as  a 
potential  hazard  for  an  inexperienced  pilot.  Most  of  the  participants  believe  that 
the  use  of  the  autopilot  will  be  required  in  some  conditions  due  to  excessive 
mental  workload. 

3.  What  are  the  similarities/differences  of  JSF  from  your  current  type 
of  aircraft? 

The  most  obvious  agreement  among  the  pilots  about  the  difference 
of  JSF  from  their  current  types  was  that  JSF  required  more  mental  effort.  The 
pilots  having  relatively  less  modern  cockpits  found  the  logic  in  operating  the 
modern  systems  easy,  and  others  who  already  operate  a  glass  cockpit  similar  to 
JSF  found  the  operational  procedures  to  be  similar  or  if  different,  easy  to  adapt. 

4.  Where  do  you  think  the  potential  areas  are  that  might  be  difficult  for 
a  transition  pilot  from  your  current  type  of  aircraft? 

The  participants  considered  managing  all  information  provided  in 
the  cockpit  to  be  a  major  challenge.  They  thought  that  this  issue  needs  to  be 
addressed  carefully  during  the  initial  phases  of  training  via  all  training  devices 
including  the  simulator. 
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5.  Generally,  is  the  data  represented  to  the  pilot  at  sufficient  level,  or 
did  it  happen  that  you  became  overwhelmed  by  over¬ 
representation/finding  and  filtering  necessary  information? 

There  is  more  information  being  presented  to  pilots  than  ever 
before,  but  the  problem  is  to  manage  that  much  information. 

6.  Considering  your  current  aircraft  type,  is  there  any  system  in  JSF 
that  serves  the  same  purpose  with  different  operation 
principles/data  representation/interface/interaction? 

No  significant  issue  reported  about  this  question. 

7.  Comparing  the  JSF  cockpit  with  your  current  type,  is  there  any 
interface/switch  that  looks  similar,  and/or  is  in  the  same  location, 
but  used  for  another  system/purpose? 

No  significant  issue  reported  about  this  question. 

8.  What  are  your  impressions  about  the  specifications  of  HMD 
(Weight,  dimensions,  Field  of  View,  contrast  and  brightness  range 
and  settings)? 

It  is  accepted  as  a  very  useful  system  helping  pilots  to  build  up  their 
SA.  The  more  experienced  pilots  consider  the  HMD  and  DAS  combined  usage 
as  a  potential  threat  due  to  some  conditions  that  might  lead  to  disorientation,  and 
thus  result  in  an  undesirable  event. 

9.  Does  the  HMD  take  your  focus/attention  from  priority  issues  and 
distract  you  from  prioritized  tasks? 

In  general,  the  HMD  symbology  did  not  prevent  participants  from 
focusing  on  events  outside  the  cockpit,  but  while  “seeing  through”  the  fuselage 
and  through  the  other  cockpit  displays,  some  participants  found  it  disorienting 
initially,  and  commented  about  being  extra  cautious  to  use  this  system  especially 
at  low  altitude  flights  and  in  bad  weather  conditions. 
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10.  What  are  your  impressions  about  the  usage  and  switching  of  HMD 
modes  and  its  data  representation? 

No  significant  issue  reported  about  this  question. 

1 1  .What  are  your  impressions  about  the  usage  and  switching  of  HMD 
modes  and  its  data  representation? 

No  significant  issue  reported  about  this  question.  Only  some  pilots 
not  familiar  to  HMD  reported  that  they  prefer  to  use  Head  Down  instruments  to 
reach  information  due  to  their  habits.  However,  they  also  stated  that  it  is  a  time 
based  issue  and  can  be  overcome  in  short  time. 

12.  What  is  your  general  impression  about  HMD?  What  are  its  specific 
strengths  and  weaknesses? 

The  general  conclusion  about  the  HMD  was  its  usefulness  for  pilots 
to  enhance  their  mission  effectiveness. 

13.  Do  you  have  any  previous  MFD  experience?  If  so,  are  there  any 
differences  (data  representation  locations,  style  and  colors)? 

The  participants  operating  MFDs  in  their  current  types  reported  that 
the  operational  logic  and  the  interface  of  similar  systems  in  the  JSF  cockpit  are 
totally  different,  thus  posing  no  NTT  issue. 

14.  What  are  your  impressions  about  the  usage  and  switching  of  MFD 
modes  and  its  data  representation? 

The  portals  (Separate  MFD  Windows  of  the  big  display  suite)  are 
easy  to  adapt  and  use.  However,  it  is  stated  that  lacking  a  thorough  training  on 
modes  and  symbology  will  prevent  pilots  from  obtaining  the  necessary 
information  when  needed. 

15.  Does  the  increased  symbology  confuse/overwhelm  you? 

The  portals  provide  perfect  SA,  and  were  found  to  be  very  useful, 
but  the  existence  of  a  lot  of  data  and  modes  was  considered  as  a  major 
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challenge  during  operations,  and  also  a  means  of  distraction  due  to  containing 
too  much  information  at  the  same  time;  thus,  it  was  seen  as  crucial  to  declutter 
the  irrelevant  information. 

16.  What  is  your  general  impression  about  MFD?  What  are  its  specific 
strengths  and  weaknesses? 

General  opinion  was  that  the  display  suite  enhances  the  pilot’s  SA 
and  is  very  powerful  as  well  as  being  useful. 

17.  What  is  your  impression  about  seat/stick/throttle  positioning  and 
their  usage? 

The  anthropometric  design  of  the  throttle,  stick  and  seat,  felt 
comfortable.  There  were  pilots  who  used  to  fly  with  a  more  straight  seat  and  stick 
in  the  middle  of  cockpit,  as  well  as  pilots  who  were  familiar  with  the  side  stick  and 
HOTAS  switchology  with  slightly  more  seat  angle  to  the  back,  yet  none 
complained  about  the  seat  and  throttle-stick  setup. 

18.  What  is  your  impression  about  switch  positioning  on  throttle/stick? 
Is  there  any  switch  that  results  in  confusion  or  contradicts  your 
previous  experience? 

The  participants  using  HOTAS  in  their  current  types  report  the 
HOTAS  in  JSF  as  easily  adaptable.  Even  if  there  were  switches  at  the  same 
location  or  the  same  purpose  but  which  operated  differently,  they  found  it  very 
easy  to  adapt.  They  expressed  the  importance  of  its  familiarization  and  training 
for  an  effective  use  because  it  controls  even  more  systems  than  it  ever  has 
before. 

19.  What  is  your  impression  about  3D  audio  in  operational  usage? 

The  3d  audio  system  was  not  used. 

20.  Are  there  any  issues  regarding  the  order  sequence  of  operational 
procedures  between  the  JSF  and  your  current  type? 


79 


The  participants  stated  that  they  did  not  have  enough  knowledge 
and  information  about  the  operational  procedures  of  JSF  to  compare  with  their 
experience.  The  tasks  they  were  supposed  to  perform  while  on  JSF  simulator 
flights  covered  limited  operational  knowledge  about  the  JSF  platform. 

21.  In  which  type  of  missions/flight  phases,  is  the  workload  of  the  pilot 
increased? 

The  pilots  reported  that  workload  was  especially  high  in  low  altitude 
missions,  which  require  a  lot  more  tasks  to  accomplish,  yet  do  not  allow  pilots  to 
use  autopilot  to  concentrate  on  tactical  tasks. 

22.  What  are  the  effects  of  increased  automation  on  your  workload,  SA 
and  flight  concentration? 

Autopilot  is  considered  to  be  very  useful.  Considering  the  high  task 
load  to  maintain  tactical  awareness  and  effectively  use  the  weapons,  pilots  stated 
that  its  use  is  necessary.  Some  participants  even  stated  it  is  compulsory  to  use 
autopilot  to  effectively  make  best  use  of  the  aircraft  systems. 

23.  What  would  you  like  to  be  changed  in  cockpit,  and  how? 

No  significant  issue  reported  about  human  factors  in  cockpit. 

24.  What  do  you  think  a  pilot  in  your  position  would  have  to  learn 
and/or  unlearn  to  fly  the  JSF? 

Some  answers  include  the  need  for  a  robust  brain-muscle 
coordination,  getting  used  to  cockpit  instruments  and  displays,  and  maybe 
starting  all  over  again  except  general  aviation  knowledge  and  skills.  These  are 
also  considered  as  the  basic  skills  to  fly  any  aircraft.  However,  besides  the 
physical  demands  of  flying  an  aircraft,  JSF  requires  a  highly  adaptable  mind  for 
new  technology,  being  able  to  filter  and  evaluate  a  large  amount  of  information. 
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B.  ANALYSIS  OF  THE  FOLLOW-ON  SURVEY  RESULTS 


1.  Multiple  Choice  Questions 

Results  from  the  multiple-choice  questions  are  shown  in  Figures  5  through 
39.  The  histograms  in  these  figures  indicate  the  agreement  level  of  participants 
with  the  related  survey  questions.  The  Y-axis  indicates  the  number  of  participants 
for  the  given  agreement  levels.  The  sum  of  the  numbers  in  some  questions  is  not 
ten  despite  the  fact  that  there  were  ten  participants.  This  is  because  some 
participants  stated  that  they  had  no  opinion  about  those  related  questions;  thus, 
they  are  not  represented  in  these  histograms.  Considering  that  all  the  histograms 
are  very  self-explanatory,  they  are  not  named  specifically. 

The  major  point  of  the  analysis  is  the  nature  of  the  data  or  answers.  One 
could  think  to  assign  numbers  between  -3,  and  3  to  represent  a  value  range  from 
“strongly  disagree”  to  “strongly  agree,”  and  then  calculate  the  average  for  each 
question.  For  instance,  a  typical  report  would  be  “The  average  agreement  level 
for  this  question  is  2.3.”  But  as  the  answers  are  categorical  (ordinal),  it  would  be 
wrong  to  apply  any  linear  mathematical  calculation  to  them.  The  moderate 
agreement  level  assigned  the  number  2  would  not  be  two  times  stronger  than  the 
agreement  level  of  “slightly  agree”  with  the  number  1.  This  is  the  reason  for 
reporting  the  agreement  levels  in  frequency  histograms.  The  histograms  provide 
the  tendencies  of  the  participants  in  each  question  graphically  and  numerically. 
After  reporting  each  question’s  histograms  without  any  conclusion,  the  discussion 
section  of  this  chapter  will  provide  the  summary  and  discussions  of  the  results, 
and  report  the  important  findings  of  the  follow-on  survey. 
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Question  1:  There  will  be  much  more  use  of  “autopilot”  in  the  JSF 
compared  to  my  current  aircraft  type. 


Question  1 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 
Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  5.  Answers  to  Question  1 


Question  2:  No  matter  how  good  the  autopilot  is,  pilots  will  still  need  to 
train  basic  flying  skills  as  much  as  previous  type. 
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Figure  6.  Answers  to  Question  2 
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Question  3:  Use  of  the  JSF  autopilot  will  greatly  help  pilots  to  focus  on  the 
tactical  situation. 


Figure  7.  Answers  to  Question  3 

Question  4:  The  autopilot  and  other  cockpit  automation  will  result  in 
possible  loss  of  situational  awareness  regarding  the  state  of  the  aircraft  position 
control  and  flight  status. 

Question  4 
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Figure  8.  Answers  to  Question  4 
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Question  5:  The  various  flight  operating  modes  in  the  JSF  are  easy  to 
learn  and  distinguish. 


Question  5 
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Figure  9.  Answers  to  Question  5 


Question  6:  It  is  easy  to  switch  between  the  modes  of  autopilot  and  transit 
from  autopilot  to  manual  flying. 


Question  6 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 
Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  10.  Answers  to  Question  6 
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Question  7;  The  pilot  can  easily  capture  any  problems  of  autopilot 
(awareness  of  aviate  &  navigate  the  A/C)  when  accomplishing  other  tactical 
tasks  in  the  cockpit. 


Question  7 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 
Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  1 1 .  Answers  to  Question  7 

Question  8:  Because  the  autopilot  of  JSF  is  highly  sophisticated  and  has 
various  modes,  basic  flying  skills  are  not  required  as  much  as  it  is  required  at 
previous  types  of  aircrafts. 


Question  8 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 
Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  12.  Answers  to  Question  8 
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Question  9:  Based  on  my  experience  in  the  JSF  simulator,  I  believe  that 
the  flight  management  system  is  easy  to  set  up  and  operate. 
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Question  9 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 
Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  13.  Answers  to  Question  9 

Question  10:  There  are  some  modes  in  the  flight  management  system  that 
I  found  difficult  to  use. 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 

Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  14.  Answers  to  Question  10 
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Question  11:  There  were  instances  that  I  encountered  when  flying  the  JSF 
simulator  for  which  I  did  not  understand  how  to  activate  or  use  the  appropriate 
operating  mode. 


Figure  15.  Answers  to  Question  1 1 


Question  12:  As  far  as  I  can  tell  there  should  be  no  difficulty  learning  how 
to  configure  the  cockpit  displays  for  flying,  navigating  and  communicating. 


Question  12 
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Figure  16.  Answers  to  Question  12 
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Question  13:  It  may  take  extra  training  time  for  pilots  to  learn  how  to 
effectively  operate  the  new  JSF  cockpit  controls  and  displays. 


Question  13 


Strongly  Moderately  Slightly  Mental  Slightly  Moderately  Strongly 

Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  17.  Answers  to  Question  13 

Question  14:  There  had  been  instances  that  I  had  to  focus  my  attention 
mostly  to  head-down  displays  to  manage  the  systems  and  reach  the  information  I 
needed. 


Question  14 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 

Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  18.  Answers  to  Question  14 
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Question  15:  Even  if  there  is  a  lot  of  information  from  various  sensors  on 
the  same  display,  I  did  not  have  any  difficulty  to  filter  and  evaluate  the  data  for 
decision  making. 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 

Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  19.  Answers  to  Question  15 
Question  16:  DAS  can  cause  disorientation  under  some  conditions. 
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Figure  20.  Answers  to  Question  16 
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Question  17:  The  idea  of  seeing  HMD  symbology  wherever  I  look  did  not 
distract  my  attention. 

Question  17 
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Figure  21 .  Answers  to  Question  17 

Question  18:  I  believe  the  HMD  failure  will  dramatically  affect  the  mission 
efficiency. 


Question  18 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 
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Figure  22.  Answers  to  Question  18 
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Question  19:  Managing  the  switches  on  throttle  and  stick  effectively  will 
require  a  considerable  amount  of  experience  and  training. 


Question  19 


Figure  23.  Answers  to  Question  19 

Question  20:  The  appeal  of  the  head-down  displays  and  the  workload 
need  to  be  done  on  those  displays  might  cause  flight  safety  issues. 


Figure  24.  Answers  to  Question  20 
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Question  21:  I  believe  that  there  needs  to  be  special  training  to  teach 
pilots  how  to  use  the  expanded  display  suite. 


Question  21 
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Figure  25.  Answers  to  Question  21 

Question  22:  Without  proper  training  and  experience  pilots  may  not  be 
able  to  handle  the  vast  amount  of  information  provided  by  the  JSF  system. 
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Figure  26.  Answers  to  Question  22 
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Question  23:  I  believe  the  main  task  of  the  pilot  will  switch  from  mostly 
flying  the  aircraft  to  making  tactical  decisions. 
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Figure  27.  Answers  to  Question  23 

Question  24:  Being  able  to  follow  the  whole  tactical  arena  did  not  affect  my 
focus  on  my  own  target/area  of  interest. 
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Figure  28.  Answers  to  Question  24 


93 


Question  25:  Managing  both  A/A  and  A/G  data  at  the  same  time  will 
overload  pilots  under  some  tactical  situations. 
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Figure  29.  Answers  to  Question  25 

Question  26:  I  felt  the  need  to  effectively  filter  and  declutter  the  presented 
information  in  most  tactical  situations. 
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Figure  30.  Answers  to  Question  26 
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Question  27:  Even  if  JSF  presents  a  very  good  tactical  picture,  a  high  level 
of  tactical  experience  is  required  to  be  able  to  use  the  capabilities  of  the  aircraft 
to  the  utmost  extent. 
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Figure  31 .  Answers  to  Question  27 

Question  28:  Compared  to  my  current  type  of  aircraft,  the  training  period 
should  be  longer  to  comprehend  the  systems  thoroughly  and  fly  the  aircraft  at  its 
capabilities. 


Question  28 


Strongly  Moderately  Slightly  Neutal  Slightly  Moderately  Strongly 

Disagree  Disagree  Disagree  Agree  Agree  Agree 


Figure  32.  Answers  to  Question  28 
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Question  29:  The  new  concept  of  JSF  requires  building  and  maintaining 
better  SA  and  more  cognitive  workload  than  my  current  type  of  aircraft. 
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Figure  33.  Answers  to  Question  29 

Question  30:  There  were  some  instances  where  I  had  difficulties  at  shifting 
my  attention  between  the  overall  tactical  picture  and  my  task  related  tactical 
picture. 
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Figure  34.  Answers  to  Question  30 
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Question  31: 1  believe  a  longer  pre-flight  preparation  is  needed  for  JSF. 
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Figure  35.  Answers  to  Question  31 

Question  32:  Even  if  the  systems  enhance  in-flight  mutual  support  at  a 
great  level,  formation  briefing  and  coordination  are  even  more  critical  than  for 
previous  types. 


Figure  36.  Answers  to  Question  32 
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Question  33:  The  simulator  flights  and  real  flights  should  be  exactly  similar 
in  terms  of  briefing,  mission  and  debriefing. 


Question  33 


Figure  37.  Answers  to  Question  33 

Question  34:  PC  trainers  donated  with  real  throttle  and  stick  controls 
would  be  significantly  beneficial  to  improve  the  systems  management  skills  of 
pilots. 


Question  34 


Figure  38.  Answers  to  Question  34 
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Question  35:  To  improve  the  pilots’  display  suite  management  and  tactical 
picture  assessment  skills,  alternative-training  systems  on  the  ground  will  be 
helpful  other  than  actual  flight  conditions. 
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Figure  39.  Answers  to  Question  35 


2.  Open-ended  Questions 

It  is  always  possible  to  miss  important  points  in  the  survey  questions. 
Another  possibility  is  that  there  are  additional  aspects  that  the  participants  can 
provide  to  the  argument.  Those  were  the  reasons  to  include  the  open-ended 
questions  to  the  follow-on  survey,  to  allow  participants  to  add  their  opinions  and 
additional  comments  about  transition,  training  and  safety  concerns  of  JSF. 

The  results  are  provided  again  as  summaries  based  on  the  common 
answers  to  each  question,  as  well  as  some  less  frequent  but  interesting  ones. 
Five  questions  were  asked,  and  the  results  can  be  seen  below. 

Question  1:  What  do  you  foresee  as  the  most  significant  problems  or 
training  issues?  Briefly  describe. 

Many  participants  saw  the  capabilities  of  JSF  as  being  significantly  better 

than  the  current  types,  and  that  there  will  be  more  demand  for  technical  and 
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tactical  knowledge  to  fly  JSF.  The  JSF  will  require  more  resources  in  training  due 
to  its  increased  capabilities;  these  will  include  such  things  as  wider  training 
areas,  ground  training  devices,  and  many  more.  Another  important  common 
opinion  is  the  importance  of  the  tactical  experience,  and  its  currency.  The 
majority  of  the  participants  agreed  that  the  transitioning  pilots  should  have 
tactical  experience  before  JSF  in  order  to  fly  safer  and  more  efficiently  due  to  its 
dominating  capabilities,  and  the  abundant  information  it  provides. 

Question  2:  Which  one  of  the  following  pilot  types  do  you  think  will  qualify 
to  effectively  and  safely  fly  all  the  missions  with  JSF  earlier  in  transition  phase:  a 
pilot  who  gained  experience  in  another  aircraft  type,  or  a  new  graduate  pilot  from 
flight  school?  Why? 

There  is  a  solid  consensus  about  this  issue,  and  the  participants  think  that 
the  JSF  is  relatively  intuitive  and  easy  to  adapt.  It  also  provides  all  and  maybe 
more  than  necessary  information  to  the  pilots,  with  its  numerous  systems,  but 
they  add  that  it  will  need  experience  to  filter  and  evaluate  the  alternatives,  in 
order  to  make  quick  and  robust  decisions.  Thus,  they  prefer  experienced  pilots  to 
be  sent  to  JSF. 

Question  3:  What  might  be  the  most  likely  cause  for  flight  safety  problems 
in  JSF? 

Two  agreements  emerge  from  this  question.  The  first  one  is  related  to 
HMD  and  DAS  related  disorientation  hazards.  The  pilots  think  that  disorientation 
is  a  potential  threat,  and  suggest  that  operating  procedures  should  be 
established  properly  and  pilots  well  trained  in  those  systems’  use.  The  second 
concern  is  the  display  suite  with  the  portals.  The  participants  report  that 
especially  in  high  workload  conditions,  fixating  on  the  display  suite  may  be  a  very 
common  and  risky  concern  about  flight  safety.  As  almost  all  of  the  mission 
related  tactical  information  is  presented  on  that  display,  there  will  be  a  tendency 
for  the  pilots  to  fixate  on  it,  and  be  unaware  or  less  aware  of  other  parameters 
and  outside  information. 
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Question  4:  What  would  be  your  recommendations  about  the  transition 
and  training  phase  of  JSF? 

The  participants  see  two  factors  as  important  in  terms  of  training  and 
transition  to  JSF.  They  want  the  training  period  to  be  very  intensive,  well  planned, 
and  also  to  have  ground  training  devices.  The  main  reason  for  this  kind  of 
intensive  training  is  the  extensive  capabilities  of  the  systems.  The  more  capable 
systems  require  more  dynamic  training.  Another  comment  the  participants  make 
is  the  importance  of  employing  experienced  pilots  in  the  transition  periods  first,  to 
be  followed  later  by  the  less  experienced  pilots. 

Question  5:  Other  Comments 

No  important  comments  were  made  for  this  question. 

C.  DISCUSSION  OF  THE  FINDINGS 

This  section  is  intended  to  summarize  the  major  points  that  emerged  from 
the  follow-on  survey.  Detailed  answers  to  the  questions  can  be  found  in  the 
previous  section  of  this  thesis. 

This  discussion  is  based  on  comments  and  responses  by  the  participants 
and  the  opinions  of  the  authors.  The  discussion  of  the  findings  is  divided  into 
three  topics:  cockpit  automation,  safety  concerns,  and  training  transition.  This 
classification  is  based  on  the  major  concerns  raised  from  the  answers,  but  also 
they  reflect  the  literature  review  chapter  of  the  thesis.  The  major  arguments 
about  modern  cockpits  from  a  human  factors  perspective  also  contain  the  same 
topics. 

As  is  thoroughly  discussed  in  the  related  work  chapter,  automation  has 
had  dramatic  impacts  on  the  human  user’s  role  and  responsibilities;  the  more 
modern,  and  complex  systems  in  the  cockpits  required  higher  levels  of  cognitive 
skills,  higher  workloads,  and  higher  training  requirements.  This  section  will  not 
provide  the  related  literature;  for  further  information  the  reader  can  refer  to 
Chapters  II  and  IV. 
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1.  Cockpit  Automation 

As  discussed  in  Chapter  IV,  automation  is  used  effectively  in  commercial 
aviation,  but  not  yet  in  military  aviation,  especially  in  fighter  cockpits.  Although 
many  individual  systems  (such  as  radars)  had  automatic  features,  the  autopilot 
has  been  rarely  used  in  the  military.  Especially  in  dynamic  situations,  military 
pilots  prefer  to  have  manual  control  of  the  aircraft.  All  these  factors  have  reduced 
the  human  factors  problems  in  fighter  cockpits  so  far.  But  the  question  of  whether 
this  will  change  in  JSF  is  raised  after  reviewing  the  design  philosophy  of  the 
aircraft.  That  question  required  further  investigation  and  was  the  reason  the 
survey  included  many  questions  regarding  this  concern. 

The  major  finding  about  this  issue  is  that  the  participants  think  the 
autopilot  will  be  used  more  than  in  their  current  aircraft.  Additionally,  they  agree 
that  the  main  task  of  the  pilot  will  shift  from  actually  flying  the  aircraft  to  making 
tactical  decisions.  These  findings  show  a  need  to  think  about  the  impacts  of 
automation  in  the  JSF  cockpit.  The  problems  and  hazards  found  in  the  literature 
will  probably  also  be  a  concern  in  fighter  cockpits  beginning  with  JSF. 
Considering  the  riskier,  more  dynamic  and  higher  workload  of  fighter  missions, 
there  is  a  need  to  understand  which  problems  will  also  be  an  issue  in  fighter 
cockpits  and  to  what  extent. 

After  acknowledging  that  the  autopilot  will  have  more  use  in  the  JSF 
cockpit,  the  participants  add  that  it  will  not  mean  that  the  basic  flying  skills  will  no 
longer  be  necessary.  The  handling  and  control  capabilities  are  expected  still  to 
be  important  skills. 

Another  concern  raised  by  the  participants  is  to  understand  if  the  autopilot 
in  JSF  is  considered  easy  to  learn  and  operate  by  the  pilots,  and  if  the  autopilot 
may  cause  the  pilots  to  lower  their  awareness  about  the  task  it  performs.  The 
main  opinion  is  that  it  will  be  easy  to  learn  and  operate,  and  to  be  aware  of  the 
modes  of  autopilot.  At  the  same  time,  some  participants  believe  that  the  use  of 
autopilot  may  decrease  the  awareness  of  the  pilots  of  the  parameters  it 
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manages.  But  they  also  think  that  the  pilots  will  be  able  to  overcome  any 
problems  about  automation  or  the  parameters  it  controls,  too.  In  conclusion, 
automated  flight  features  may  be  easy  to  learn  and  operate,  but  even  if  there  is 
slight  agreement  on  the  decrease  in  awareness,  it  is  expected  that  the  pilots  will 
be  able  to  overcome  any  related  problems.  Considering  that  the  real  missions  will 
be  more  stressful  and  demanding,  the  authors  think  that  these  concerns  about 
cockpit  automation  require  further  research  for  more  precise  understanding,  and 
may  pose  a  higher  threat  than  indicated  by  the  participants’  expectations. 

The  last  agreement  about  automation  is  that  it  does  what  is  claimed  in  the 
JSF  cockpit.  The  participants  agree  that  the  autopilot  and  other  automated 
systems  helped  them  to  focus  more  on  the  tactical  picture.  Other  important 
aspects  of  this  finding  will  be  discussed  in  the  next  section. 

2.  Tactical  Decision  Making  and  Systems  Management 

This  section  contains  a  wide  spectrum  of  human  factors  areas,  such  as 
SA,  workload,  spatial  disorientation,  and  cognitive  tunneling.  Many  question  were 
asked  about  the  use  of  the  systems,  workload,  safety  traps,  and  other  potential 
issues. 

One  of  the  objectives  of  the  survey  was  to  capture  the  opinions  of  the 
participants  about  the  usability  of  the  cockpit  controls  and  displays.  The  concern 
here  is  not  to  capture  the  workload  and  stress  during  the  missions,  but  whether  it 
is  easy  to  operate  the  systems,  such  as  changing  modes,  making  modifications 
and  managing  the  menus.  There  is  a  general  agreement  that  the  operation  of  the 
flight  management  systems  is  easy.  The  participants  did  not  find  that  there  were 
modes  that  were  hard  to  understand.  The  participants  indicated  that  it  will  be 
easy  to  learn  how  to  operate  those  systems  and  that  the  management  system 
will  probably  not  be  a  major  problem. 

There  was  no  general  consensus  about  whether  it  will  take  extra  time  to 
learn  to  effectively  operate  the  controls.  However,  when  the  question  is  related  to 
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the  control  stick  and  throttle,  and  special  emphasis  is  given  to  the  display  suite, 
the  participants  think  that  they  should  be  addressed  very  carefully  and  focus  on 
using  all  necessary  training  equipment. 

Both  ample  information  from  various  sensors,  and  possible  tactical 
changes  in  air  combat  could  present  higher  workloads  to  pilots.  Additionally,  the 
big  LCD  suite  will  be  the  main  display  while  performing  tactics  in  those  high 
workload  situations.  If  the  autopilot  is  also  used  in  these  situations,  awareness, 
cognitive  tunneling,  and  other  related  hazards  might  be  a  potential  threat.  To 
investigate  these  types  of  threats,  the  survey  included  two  groups  of  questions. 

The  first  group  of  questions  addressed  the  concern  that  the  participants 
had  times  when  they  needed  to  focus,  or  fixate  on  the  display  suite.  The  opinion 
is  that  there  is  a  need  for  using  the  display  suite  in  many  tactical  situations.  The 
participants  agreed  that  there  is  a  reliance  on  the  display  suite  in  general  too.  It  is 
legitimate  to  say  that  there  is  no  doubt  in  the  participants’  answers,  that  the  pilots 
will  be  using  the  display  suite  very  frequently  in  many  tactical  situations. 

After  assuming  the  fact  that  the  pilots  will  focus  mostly  on  the  display 
suite,  the  following  question  is  whether  it  will  pose  a  threat  to  flight  safety.  And 
figuring  out  this  issue  was  the  objective  of  the  second  mentioned  group  of 
questions.  Some  of  the  related  answers  are  that  there  were  instances  when  the 
pilots  think  they  focused  on  the  display  suite  more  than  they  think  is  safe,  and 
they  generally  think  there  is  a  risk  that  the  pilots  will  fixate  on  this  display  suite 
and  become  unaware  of  other  information  such  as  “fuel  state”,  “altitude 
limitations”  and  many  more.  Especially  in  higher  workload  situations,  the 
participants  report  that  they  needed  to  focus  more  on  the  displays  in  order  to 
reach  and  evaluate  the  tactical  information. 

According  to  the  participants,  the  threat  mentioned  above  is  obvious,  but 
with  the  more  specific  questions  about  the  display  suite,  there  are  blurred  areas. 
The  authors  expected  to  find  data  filtering  difficulties,  but  the  participants 
reported  that  even  if  there  was  a  great  deal  of  information,  they  did  not  have 
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difficulties  in  reacting  appropriately.  Similarly,  no  significant  agreement  is 
reported  about  the  existence  of  both  Air-to-Air  and  Air-to-Ground  symbology  at 
the  same  time.  Roughly,  half  of  the  participants  report  that  it  may  overload  the 
pilots,  and  the  remaining  half  do  not  agree.  Additionally,  they  also  did  not  agree 
clearly  on  the  need  to  declutter  some  symbology. 

Another  controversial  issue  is  whether  the  pilots  will  have  attentional  focus 
problems  due  to  the  fact  that  they  will  be  presented  with  a  very  big  picture  of  the 
battle  area.  For  instance,  a  flight  leader  may  be  responsible  to  intercept  an 
enemy  formation,  but  in  JSF  he  will  have  almost  all  battlefield  information  in  front 
of  him,  such  as  other  friendly  assets  and  both  air  and  ground  threats.  After  a 
while,  he  should  focus  on  building  intercept  geometry,  target  sorting,  and 
monitoring  the  target  formation  in  more  detail.  Whether  other  “big  picture” 
information  will  distract  him  from  focusing  on  his  target  is  an  issue.  Will  the  pilot 
be  able  to  focus  his  attention  back  and  forth  between  his  targets  and  the  general 
picture?  The  participants’  answers  reveal  that  they  did  not  have  any  difficulty  in 
focusing  on  their  target  areas;  however,  half  of  them  experienced  instances 
where  they  had  difficulties  in  shifting  their  attention  between  the  overall  tactical 
picture  and  their  task  related  picture. 

Surprisingly,  the  participants  did  not  have  any  major  difficulty  in  filtering 
the  data,  managing  both  Air-to-Air  and  Air-to-Ground  data  simultaneously,  and 
operating  the  display  suite.  Additionally,  they  did  not  report  any  apparent 
problems  in  shifting  attentional  focus,  and  they  expressed  that  it  was  easy  to  set 
up  and  operate  the  systems.  However,  when  asked  whether  a  high  level  of 
tactical  experience  is  required  to  use  the  capabilities  of  JSF  to  the  utmost  extent, 
the  participants  answered  in  the  affirmative.  They  had  many  comments 
addressing  the  importance  of  the  tactical  experience.  The  participants  agree  that 
filtering  and  evaluating  the  data,  as  well  as  making  quick  and  proper  decisions 
will  require  tactical  experience.  The  reason  for  the  contrary  aspect  can  be 
twofold.  Either  the  participants  found  the  use  of  the  display  suite,  attentional 
focus,  and  data  filtering  not  to  be  a  problem,  because  they  could  cope  with  many 
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issues,  given  their  experience,  or  they  see  the  complex  cognitive  processes  while 
decision  making  and  managing  the  displays,  and  processing  information,  as 
separate.  And  they  think  that  while  the  former  task  is  hard,  the  second  tasks  are 
relatively  easier.  Either  way,  the  experience  factor  cannot  be  denied,  but  the  data 
filtering  and  display  suite  operation  in  actual  missions  require  further  research. 

Disorientation  and  distraction  were  also  the  concerns  of  this  thesis.  Both 
are  related  to  the  visual  systems,  especially  HMD  and  DAS,  as  the  authors 
predicted.  The  participants  do  not  support  the  first  suspicion  of  the  authors,  that 
the  HMD  will  distract  the  pilots’  attention  with  the  symbology  it  presents 
continuously.  They  report  that  seeing  HMD  symbology  wherever  they  look  did  not 
distract  their  attention.  But  the  major  potential  hazard  expected  by  the 
participants  is  the  DAS,  if  the  pilot  is  not  trained  to  use  it  properly.  The 
participants  agree  that  the  DAS  may  cause  disorientation  in  some  conditions. 
Additionally,  most  of  them  mentioned  the  same  risk  in  their  open-ended  answers. 
They  also  commented  that  the  training  and  operational  procedures  of  this  system 
should  be  established  very  carefully. 

3.  Transition  Training 

Among  the  areas  investigated  in  the  survey,  the  concerns  related  to 
training  during  the  transition  period  showed  the  strongest  agreement  among  the 
participants.  The  survey  explored  the  opinions  of  the  participants  about  the 
training  in  two  major  topics.  In  the  first  one,  the  systems’  training  issues  are 
questioned,  whereas  in  the  other  one,  the  training  system  is  investigated  as  a 
whole. 

The  systems  of  interest  in  this  context  are  HOTAS,  the  display  suite,  and 
DAS.  The  survey  has  explicit  questions  for  the  first  two  systems,  and  the 
recommendations  about  DAS  training  came  mostly  from  the  answers  related  to 
safety  concerns.  The  reports  about  DAS  are  mentioned  in  the  previous  section, 
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and  the  emerged  agreement  is  that  it  may  be  prone  to  cause  disorientation  in 
some  conditions,  and  properly  should  be  addressed  during  training  to  mitigate 
any  potential  hazard. 

The  initial  thought  of  the  authors  was  that  pilots  would  need  more 
emphasis  on  training  for  HOTAS  management.  HOTAS  is  not  new  to  the  cockpits 
and  pilots,  but  the  reason  is  that  the  JSF  has  more  systems  with  enhanced 
capabilities,  and  thus  more  switches  and  menus  in  HOTAS.  Additionally,  the 
pilots  may  have  a  higher  workload  in  order  to  use  all  capabilities  of  such  an 
advanced  fighter.  However,  the  general  opinion  of  the  participants  is  neutral  on 
the  question  of  whether  HOTAS  management  will  demand  considerable  amounts 
of  experience  and  training,  but  they  mostly  agreed  that  it  will  be  very  useful  to 
have  a  PC-Trainer  type  of  device  with  real  HOTAS  to  train  pilots  about  the 
management  of  the  system.  When  it  comes  to  the  training  of  the  display  suite, 
there  is  a  stronger  agreement.  The  participants  mostly  agree  that  special 
emphasis  should  be  given  to  display  suite  management  during  training. 
Additionally,  evaluating  the  information  from  the  displays  for  tactical  assessment 
was  also  found  to  be  an  important  training  issue,  and  participants  strongly  agreed 
that  alternative  training  devices  such  as  PC-Trainers  will  be  very  beneficial  for 
this  purpose.  Another  supporting  fact  for  the  opinion  of  using  alternative  training 
devices  is  that  the  participants  also  mentioned  it  in  their  open-ended  answers. 
Addressing  the  same  cognitive  processes  while  pilots  make  complex  decisions  in 
very  demanding  situations  during  ground  training  looks  to  be  more  important  in 
JSF  than  in  the  current  types. 

Another  aspect  of  interest  about  the  training  is  the  general  opinions  of  the 
participants.  The  most  general  question  related  to  this  issue  is  whether  JSF  will 
require  a  longer  training  period  than  the  current  aircraft  to  make  pilots  combat 
ready.  Whereas  the  general  tendency  was  to  agree  this  statement,  three  out  of 
ten  pilots  disagreed.  One  of  the  critical  factors  of  the  participants’  JSF 
background  is  that  they  did  not  fly  missions  according  to  any  performance 
grading  criteria,  but  just  experienced  various  missions  and  gave  feedback.  Both 
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during  actual  JSF  training,  exercise  and  combat  missions,  the  pilots  will  have  real 
objectives  that  will  be  measured  objectively.  It  is  highly  probably  that  the  more 
demanding  and  risky  missions  could  give  the  participants  a  better  perspective, 
and  they  could  be  able  to  see  the  difficulty  levels  about  the  tasks  better,  and 
more  realistically. 

While  slightly  agreeing  that  JSF  should  have  a  longer  training  period,  the 
participants  strongly  agree  that  longer  preflight  preparation  will  be  needed  for 
JSF  missions.  Moreover,  they  also  strongly  agree  that  even  if  the  systems 
enhance  mutual  support  to  a  large  degree,  formation  briefing  and  coordination 
will  be  even  more  critical  than  in  current  fighters.  Another  agreement  is  that  the 
participants  think  that  the  simulator  missions  should  be  as  similar  as  possible  to 
actual  missions.  All  of  the  aforementioned  strong  agreements  mean  that  the 
participants  think  the  missions  in  JSF  will  be  more  demanding,  and  thus  require 
longer  preparation,  more  coordination,  briefing  and  more  simulator  practices. 
Altogether,  these  comments  indicate  a  need  for  longer  and  more  demanding 
training  in  JSF. 

An  interesting  point  emerged  among  the  answers,  in  that  the  participants 
also  agreed  about  the  role  shift  phenomenon.  Some  of  them  commented  that  the 
training  should  focus  more  on  data  filtering  and  decision-making  in  complex 
situations  skills.  These  skills  are  also  important  for  the  current  types,  but  with  the 
increasing  amount  of  data,  enhanced  capabilities,  and  possibly  harder  demands 
during  JSF  missions,  these  skills  may  become  more  important  than  before,  and 
should  be  specifically  addressed  during  the  curriculum. 

The  participants  agreed  strongly  about  several  aspects  of  the  transition 

period:  The  training  devices  such  as  simulators  and  PC-Trainers  should  be  used 

to  the  utmost  extent;  transition  pilots  should  have  tactical  experience  before  JSF. 

The  main  reason  is  that  the  participants  think  the  experience  is  the  crucial  factor 

for  evaluating  the  data,  and  using  it  for  making  tactical  decisions  in  JSF.  In  order 

to  cope  with  abundant  data,  and  high  workload,  the  participants  think  experience 

is  critical.  Especially,  the  consensus  is  that  the  first  group  of  transition  pilots 
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should  be  very  carefully  selected  from  among  the  experienced  pilots,  and  two 
participants  think  that  the  less  experienced  pilots  could  be  also  transitioned 
directly  to  JSF  after  the  first  group,  but  with  a  well  researched,  and  developed 
instructional  curriculum.  The  participants  agree  that  the  pilots  should  be  trained 
thoroughly  for  each  difficulty  during  ground  training  to  prepare  them  to 
demanding  situations  during  actual  missions.  The  issues  include  tactical 
decision-making,  using  both  A-A,  and  A-G  capabilities  simultaneously, 
information  reaching  and  evaluation  capabilities,  and  switchology. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY  OF  KEY  FINDINGS 

Although  there  are  more  concerns  that  emerged  from  the  research  of  this 
thesis,  this  chapter  will  focus  on  the  ones  most  agreed  upon  by  the  participants. 
Some  of  these  predictions  are  also  strongly  supported  by  the  literature.  Findings 
about  automation  and  display  suite  operations  are  examples  in  this  category. 
Other  findings  cannot  be  directly  found  in  the  literature,  but  not  because  they  do 
not  have  scientific  support.  The  main  reason  is  that  they  are  related  to  the  new 
systems  in  the  cockpits  such  as  DAS.  Although  there  are  many  studies  about 
HUD  and  HMD,  the  DAS  has  many  unique  capabilities  that  prevent  using  the 
findings  of  HUD  and  HMD  to  solve  the  potential  problems  in  DAS.  The  potential 
concerns  are  given  in  a  summary  sentence  format;  for  more  detail,  the  previous 
chapter  will  provide  enough  information. 

1.  Increasing  Use  of  Autopilot 

The  need  to  use  the  autopilot  at  a  significantly  increasing  level  is  one  of 
the  findings  of  the  follow-on  survey.  The  participants  also  acknowledged  the  role 
shift  of  the  pilot  due  to  increased  automation.  Even  though  participants  did  not 
directly  point  out  a  problem  related  to  mode  or  system  awareness  during 
simulator  missions,  the  authors  believe  that  the  findings  in  the  literature  about 
increasing  autopilot  use  will  be  valid  and  applicable  to  JSF.  Most  of  the  studies  in 
the  literature  examined  autopilot  use  in  commercial  aircraft.  Even  though  the 
tasks  and  required  pilot  skills  vary  from  commercial  cockpits  to  fighter  cockpits, 
the  common  human  factors  issues  also  will  be  experienced  in  the  JSF  cockpit. 
The  JSF  pilot  will  have  more  tasks  at  hand,  that  are  to  be  accomplished  in  a 
much  more  stressful  environment,  and  with  greater  demand  on  cognitive 
resources;  thus,  the  authors  predict  that  JSF  pilots  will  also  be  prone  to 
automation-related  human  factors  concerns. 
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2.  HMD  and  DAS  Usage 

Throughout  aviation  history,  it  has  been  frequently  observed  that  while 
technology  and  inventions  solve  the  current  problems,  they  also  give  rise  to  new 
human  factors  concerns.  DAS  represents  a  totally  new  concept  in  the  fighter 
cockpit,  and  the  participants  strongly  agree  that  it  may  cause  disorientation  in 
some  cases.  They  think  that  the  operational  procedures  and  DAS  usage  in 
different  environmental  and  meteorological  conditions  should  be  thoroughly 
examined  and  addressed  in  pilot  training. 

3.  Display  Suite  Management 

The  display  suite  in  JSF  is  the  most  noticeable  component  of  the  cockpit. 
This  display  will  be  the  prominent  PVI  element  during  flights,  where  the  pilot  can 
reach  all  the  sensor  data  and  interact  with  the  systems  as  required.  The  features 
of  the  previous  aircraft  displays  are  combined  into  this  display  in  JSF,  thus 
making  the  display  more  appealing  and  prone  to  visual  and  cognitive  tunneling 
issues  as  seen  in  the  literature.  The  participants  also  agree  that  this  display  suite 
may  cause  cognitive  tunneling,  and  fixation  issues  during  operations. 

4.  Tactical  Experience  and  Decision  Making 

The  participants  strongly  agreed  that  tactical  experience  is  one  of  the  most 
vital  features  for  a  pilot  to  make  the  most  out  of  the  systems  in  JSF.  Even  though 
filtering  the  data  from  various  sensors  is  not  thought  to  be  difficult  by  the 
participants,  assessing  this  data  in  order  to  make  demanding  tactical  decisions 
will  require  significant  tactical  experience.  Considering  the  changing  mission 
concept  of  JSF,  combined  Air-to-Air  and  Air-to-Ground  data,  and  new  systems, 
the  authors  agree  with  participants  on  the  importance  of  tactical  experience. 
Dealing  with  more  systems  and  data  with  the  same  attentional  resources  will  be 
one  of  the  crucial  aspects  of  the  required  skills  in  JSF.  The  participants  not  only 
stated  the  importance  of  tactical  experience  in  regular  missions,  but  also  during 
transition  and  follow-on  training  phases. 
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B.  TRAINING  OBJECTIVES 


After  identification  of  the  major  concerns,  there  is  a  need  to  address  them 
during  training  and  while  establishing  operational  procedures  of  the  aircraft. 
These  two  processes  are  closely  related;  if  there  is  an  operational  procedure,  the 
system  trains  the  trainees  to  accomplish  that  procedure.  This  section  includes 
the  training  objectives  that  are  thought  to  mitigate  any  potential  problem  related 
to  the  identified  concerns.  The  training  objectives  are  created  to  meet  the 
common  problems  found  by  the  literature,  and  predicted  by  this  study.  The 
training  objectives  do  not  cover  all  of  the  required  objectives  related  to  each 
predicted  concern,  but  just  the  major  findings  of  this  thesis. 

1.  Increasing  Use  of  Autopilot 

•  The  pilots  should  have  a  thorough  understanding  of  the  automation 
technical  data,  operation  and  modes,  and  their  interactions  with 
other  systems. 

•  The  pilots  should  perceive  automation  as  another  important  system 
of  the  aircraft  and  build  skills  for  robust  coordination  and  operation. 

•  The  pilots  should  be  aware  of  the  tasks  performed  by  autopilot  and 
autopilot  modes  continuously,  even  under  high  workload  conditions. 

•  The  pilots  should  also  be  able  to  solve  the  emergency  situations 
related  to  automation  malfunctions  and  while  automation  controls 
the  parameters. 

2.  HMD  and  DAS  Usage 

•  The  pilots  should  build  robust  mental  models  of  HMD  and  DAS  and 
their  operational  procedures. 
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The  pilots  should  be  ready  to  accomplish  their  tasks  during  HMD 
failure  successfully  and  they  should  cross  check  the  parameters 
shown  in  HMD  with  Head-Down  indications  as  required.  They 
should  not  over  trust  the  HMD. 

The  pilots  should  learn  to  transition  safely  back  and  forth  between 
Head-Up  and  Head-Down  operations.  In  addition  to  Head-Up  to 
Head-Down  transitions,  the  DAS  transitions  are  predicted  to  be 
critical  phases  of  the  flight.  The  pilots  will  “see  through”  the 
fuselage  and  look  for  targets  of  interest  under  the  aircraft,  etc.,  and 
then  they  will  have  to  transition  back  to  the  normal  vision  or  vice 
versa.  Problems  such  as  disorientation  and  attentional  shift  are 
predicted,  especially  when  these  transitions  happen  during  high 
workload,  adverse  weather,  and  other  demanding  situations.  The 
pilots  should  be  able  to  use,  and  transition  to  or  from  the  DAS 
safely.  Another  aspect  of  this  issue  is  the  need  for  well-defined 
operational  procedures  that  will  enable  the  pilots  to  comprehend 
the  current  situation,  and  make  proper  assessments  depending  on 
the  operational  procedures. 

Display  Suite  Management 

The  pilots  should  acquire  full  knowledge  about  the  display  suite. 
The  raw  technical  information,  modes,  and  menus  of  the  portals, 
meanings  of  the  symbology,  and  the  required  skills  for  operational 
procedures  are  among  the  important  aspects  related  to  the  display 
suite  management. 

The  pilots  should  be  able  to  setup,  operate  and  modify  the  portals 
effectively  according  to  the  mission  requirements. 

The  pilots  should  cross  check  the  data  shown  with  various  sensors 

if  possible,  and  be  able  to  anticipate  any  problems  about  the 

symbology  and  presented  information. 

114 


•  The  pilots  should  be  ready  to  accomplish  their  tasks  successfully 
during  display  or  portal  failure. 

•  The  pilots  should  not  fixate  on  the  display  even  under  high 
workload  tactical  situations,  and  should  also  be  aware  of  other 
parameters. 

•  The  pilots  should  build  comprehensive  knowledge  about  the 
interactions  and  cooperation  among  the  systems  feeding  the 
display  suite.  In  case  of  a  failure  of  any  of  those  systems,  the  pilots 
should  be  aware  of  the  reliability  of  the  information  at  hand  and 
make  correct  decisions  in  these  contingency  situations. 

4.  Tactical  Experience  and  Decision  Making 

•  The  pilots  should  be  proficient  enough  to  filter  the  unnecessary 
data  easily,  and  to  locate  the  required  information  in  the  tactical 
displays. 

•  The  pilots  should  be  able  to  process  and  evaluate  the  presented 
data  quickly  and  accurately. 

•  The  pilots  should  have  proper  prioritization  and  attention  allocation 
skills  and  techniques  for  highly  demanding  and  high  workload 
situations. 

•  The  capable  and  modern  sensors  will  flood  the  pilots  with  a  great 
deal  of  information,  and  it  is  likely  that  the  pilots  will  experience 
many  situations  that  are  complex,  hard  to  analyze,  and  in  which  it  is 
hard  to  make  proper  decisions.  The  pilots  should  be  capable  of 
making  quick  and  accurate  decisions  under  such  complex 
scenarios. 
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C.  GENERAL  TRAINING  GUIDE 

The  predictions  of  this  thesis  are  parallel  to  the  research  literature. 
Important  aspects  of  modern  glass  cockpit  aircraft  are  generally  similar  in  terms 
of  the  human  factors  perspective.  Increasing  automation,  LCD  or  MFD  displays, 
increasing  sensors  with  enhanced  capabilities — all  present  new  challenges  to 
pilots.  Thus,  the  authors  think  that  the  solutions  proposed  in  the  literature  will 
also  be  applicable  to  the  JSF  case.  The  limitations  explained  in  previous 
chapters  prevented  conduct  of  actual  experiments;  therefore,  the  basis  for  this 
chapter  is  the  research  literature,  participants’  comments  and  the  experience  of 
the  authors. 

1.  Increasing  Use  of  Autopilot 

Objective  1.1  The  pilots  should  have  a  thorough  understanding  of  the 
automation  technical  data,  operation  and  modes,  and  its  interactions  with  other 
systems. 

Objective  1.2  The  pilots  should  perceive  automation  as  another  important 
system  of  the  aircraft,  and  build  skills  for  robust  coordination  and  operation. 

One  of  the  findings  supporting  the  importance  of  these  objectives  is  that 
Sarter  (2000)  reports  that  pilots  do  not  have  a  through  understanding  of  the 
structure  and  operational  procedures  of  automated  systems. 

Rigner  and  Dekker  (2000)  are  among  the  researchers  proposing 
incorporation  into  pilot  training  of  a  curriculum  dedicated  to  automation.  They 
note  that  automation  training  is  given  in  transition  training  today,  and  shifting  that 
to  earlier  fundamental  training  may  be  a  better  approach. 

Rigner  and  Dekker  (2000)  also  observed  that  the  content  of  the  current 
automation  training  is  incomplete.  The  training  is  more  based  on  general 
technical  information  about  the  automated  systems  and  very  limited  scenarios. 
They  claimed  that  automation  should  be  introduced  and  taught  as  a  team 
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member  of  the  cockpit  rather  than  a  subsystem  of  the  cockpit  systems.  This  is 
the  case  in  most  of  the  phases  of  flight,  and  autopilots  make  decisions  and  give 
inputs  as  the  pilots  do. 

Casner  (2003)  claimed  that  almost  all  current  aircraft  benefit  from 
automation,  and  that  this  topic  can  therefore  be  addressed  in  basic  flight  training. 
This  stage  is  where  pilot  trainees  get  their  introduction  to  general  topics  about 
aviation  such  as  weather,  aerodynamics,  flight  rules  and  many  others.  They  are 
not  assigned  to  their  final  aircraft  type  yet,  and  usually  fly  their  first  aircraft  in  their 
careers.  However,  this  is  a  suitable  time  to  train  them  about  the  fundamentals  of 
cockpit  automation.  Currently,  the  automation  training  varies  from  type  to  type. 

To  study  the  subject  further,  Casner  (2003)  conducted  an  experiment  to 
measure  the  effectiveness  of  such  a  curriculum.  In  addition  to  a  control  group, 
the  target  group  received  the  aforementioned  training  in  a  small  aircraft  cockpit, 
and  the  target  group  demonstrated  positive  training  transfer  in  their  second  trials 
in  a  commercial  aircraft’s  simulator  while  performing  automation-related  tasks. 

The  fighter  pilots  scheduled  to  fly  JSF  should  be  trained  on  the 

fundamentals  of  automated  systems,  and  especially  autopilots  in  their  initial  pilot 

training,  and  then  build  their  professional  skills  with  their  aircraft-specific  type 

trainings.  Where  this  solution  is  not  applicable,  this  training  can  be  implemented 

within  the  JSF  training  itself.  Either  the  fundamentals  and  basic  principals  may  be 

given  before  proceeding  with  the  training  of  automated  systems  of  JSF,  or  they 

may  be  directly  integrated  to  JSF  autopilot  systems  training.  At  a  first  glance,  the 

second  way  sounds  more  practical,  but  further  research  is  required  in  order  to 

figure  out  the  better  way  scientifically.  The  objective  of  this  change  is  to  provide 

pilots  better  background  about  cockpit  automation,  and  thus  help  them  build 

more  robust  mental  models  about  all  automated  systems  in  the  JSF  cockpits. 

During  this  training  phase,  another  important  objective  should  be  to  help  pilots 

build  robust  perception  about  the  autopilot.  Considering  that  the  pilots  may  use 

the  autopilot  in  demanding  situations,  it  is  crucial  to  give  them  the  required  skills 

for  effective  cooperation.  The  pilots  should  be  directed  to  not  perceive  the 
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autopilot  as  a  “fire  and  forget”  system,  but  rather  as  a  separate  system  to  be 
cooperated  with,  and  checked  for  proper  operation  when  necessary.  The 
recommended  topics  for  the  automation  ground  training  for  JSF  pilots  are  as 
follows: 

Recommended  Initial  Automation  Training  Topics 

•  The  Basic  Structure  and  Principals  of  Autopilot  Systems 

•  Common  Problems  Found  by  Literature  Concerning  Cockpit  Automation 

•  The  Interactions  and  Interrelations  between  the  Autopilot  and  Other 

Systems  in  JSF 

•  System  Analysis  and  Problem  Solving  of  Automation  in  JSF 

•  Analysis  of  the  Autopilot  Modes  in  JSF  and  Potential  Threats  During 

Missions 

•  Pilot-Autopilot  Cooperation  Procedures 

Objective  1.3  The  pilots  should  be  aware  of  the  tasks  performed  by 
autopilot  and  autopilot  modes  continuously,  even  under  high  workload  conditions. 

Objective  1.4  The  pilots  should  be  able  to  solve  the  emergency  situations 
related  to  automation  malfunctions  and  while  automation  controls  the 
parameters. 

The  findings  of  Sarter  and  Woods  (1994)  supported  the  common  fact 
about  pilots  on  automation.  They  report  that  pilots  were  comfortable  in 
performing  the  standard  basic  tasks  in  cockpits  such  as  “intercepting  a  radial, 
building  or  executing  a  holding  pattern...,”  but  they  experienced  difficulties  during 
the  tasks  requiring  comprehensive  understanding  about  automation  such  as 
“aborting  a  takeoff  at  40  kts  with  autothrottles  on”  and  “...anticipating  when  go- 
around  mode  becomes  armed  through  landing...”  (p.  14). 
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Similarly,  the  findings  of  Sarter,  Mumaw,  and  Wickens  (2007)  were  also 
consistent  with  the  previous  studies.  During  their  research,  both  with  subjective 
and  objective  measures,  the  pilots  allocated  considerably  more  of  their 
attentional  resources  on  basic  flight  parameters  than  on  the  automated  systems. 
Their  automation  awareness  was  much  lower  than  their  general  awareness,  and 
one  of  the  reasons  was  reported  as  improper  mental  models  about  the 
automation. 

There  are  two  major  points  related  to  Objectives  3  and  4:  the  awareness 
problem,  and  the  lack  of  skills  to  solve  complex  situations.  A  broader  discussion 
about  the  decision-making  skills  in  complex  situations  will  be  given  in  the  Tactical 
Experience  section  of  this  chapter,  and  the  emergency  situation  solving  skills 
about  the  automation  are  the  major  concern  about  the  automation  training 
objectives. 

The  recommendations  concerning  the  above  issues  are  as  follows: 

Increasing  the  Awareness  and  Enhancing  The  Complex  Problem  Solving 
Skills  of  Pilots  about  Cockpit  Automation 

•  Develop  the  Operational  Procedures  so  that  they  make  sure  the  pilots  also 
control  the  automation-related  parameters  and  modes  as  they  do  with 
other  flight  parameters. 

•  Ensure  that  pilots  acquire  those  skills  mentioned  above  during  ground 
training,  especially  in  simulator  missions;  set  sortie  objectives  related  to 
the  awareness  of  automation  in  JSF  cockpit. 

•  In  simulator  missions,  stress  the  emergency  solving  capabilities  of  pilots  in 
two  conditions:  the  autopilot  is  controlling  some  parameters,  and  any  other 
system  failed,  causing  an  emergency  situations,  or  again  the  autopilot  is 
controlling  some  parameters,  and  it  itself  fails  causing  an  emergency 
situation.  Inject  these  kinds  of  events  to  the  simulator  missions  also  under 
high  workload  situations,  while  pilots  are  busy  with  tactical  decisions,  and 

tend  to  forget  to  check  what  the  autopilot  is  doing. 
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2.  HMD  and  DAS  Usage 

The  results  of  both  interviews  and  surveys  do  not  indicate  any  major 
problem  with  HMD  use.  In  contrast,  the  situation  changes  with  DAS  and  the 
participants  agree  that  DAS  operational  procedures,  and  training  curriculum 
should  be  specifically  established  in  order  to  mitigate  any  disorientation  or 
attentional  focus  concerns.  Even  though  there  have  been  many  studies 
conducted  about  the  HMD  and  HUD  in  fighter  cockpits,  the  DAS  is  a  brand  new 
system.  The  authors  were  not  able  to  locate  any  studies  about  DAS  use  in 
cockpits;  thus,  the  predictions  by  the  participants  are  the  main  resource  to 
anticipate  any  problems,  as  well  as  to  propose  solutions. 

The  training  objectives  regarding  HMD  use  mostly  address  the 
effectiveness  issues,  but  the  DAS  training  objectives  are  more  related  to  flight 
safety;  thus,  they  are  more  critical. 

Objective  2. 1  The  pilots  should  build  robust  mental  models  of  HMD  and 
DAS,  and  their  operational  procedures. 

Because  the  DAS  is  a  new  system,  there  is  no  coverage  of  related 
systems  in  basic  flight  training  to  provide  a  framework  for  the  JSF  pilots’  DAS 
training.  Pilots  do  not  need  deep  knowledge  of  the  technological  aspect  of  DAS, 
but  the  focus  is  more  on  the  operational  procedures.  The  operational  procedures 
and  the  safety  concerns  with  DAS  should  be  analyzed  in  high  detail.  This  is  the 
first  step  by  the  instructional  designers  and  the  frontiers  of  the  JSF  before  it  finds 
its  way  to  air  forces.  Then,  the  second  important  step  is  the  training  of  the  pilots. 
Given  the  fact  that  their  experiences  will  not  include  a  system  similar  to  DAS, 
they  should  be  all  trained  about  the  anticipated  safety  issues  such  as  transitions 
back  and  forth  in  adverse  weather  conditions,  and  the  normal  procedures.  The 
objective  with  these  training  topics  is  to  prevent  any  mishaps  that  could  happen 
because  the  pilots  do  not  follow  the  operational  procedures  or  use  DAS  in 
conditions  where  they  should  not  use  it.  They  also  should  be  trained  about  the 
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scientific  fundamentals  of  disorientation  or  attentional  focus  problems  possible 
with  DAS  use.  Recommendations  are  as  follows: 

The  Recommended  Topics  About  Das  Ground  Training 

•  The  Capabilities  and  Limitations  of  DAS 

•  Emphasis  on  the  Standard  Operational  Procedures 

•  Human  Factors  Concerns  and  Potential  Threats  About  DAS 

(Disorientation,  Attentional  Focus) 

•  The  Use  of  DAS  in  Adverse  Conditions 

•  DAS  Emergency  and  Recovery  Procedures 

Objective  2.2  The  pilots  should  be  ready  to  successfully  accomplish  their 
tasks  during  HMD  failure,  and  they  should  cross  check  the  parameters  shown  in 
HMD  with  Head-Down  indications  as  required.  They  should  not  over  trust  the 
HMD. 

One  of  the  important  projections  of  this  study  is  the  potential  effect  of  HMD 
on  mission  efficiency.  The  HMD  is  predicted  to  be  very  helpful  to  the  pilots  during 
their  missions.  Thus,  there  may  be  a  risk  to  over  trust  the  HMD  and  the  pilots 
may  choose  to  accomplish  their  missions  using  the  HMD  more  than  they  should. 
There  are  two  crucial  points  about  this  concern.  The  first  one  is  that  the  pilots 
may  not  crosscheck  HMD  indications  with  the  Head-Down  displays.  This  is  a 
common  error  in  current  fighters.  The  pilots  are  instructed  to  use  the  Head-Down 
displays  as  their  main  displays  and  crosscheck  the  HUD  or  HMDs  with  Head- 
Down  displays  while  they  shift  their  focus  to  HUD  or  HMD.  The  authors  do  not 
have  the  technical  information  about  the  HMD  system  of  JSF.  Unless  the 
designers  eliminated  any  possibility  that  the  HMD  can  display  different 
information  than  the  Head-Down  displays,  this  issue  will  continue  to  be  crucial  in 
terms  of  efficiency  and  flight  safety. 

The  second  important  aspect  is  the  skills  of  the  pilots.  While  over-using 

HMD,  they  may  degrade  their  capabilities  to  accomplish  their  missions  using 
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other  displays  or  systems,  and  in  case  of  a  HMD  failure,  their  efficiency  may  be 
affected  dramatically.  Unless  the  operational  procedures  will  dictate  aborting  the 
missions  in  case  of  HMD  failure,  this  will  also  be  one  of  the  important  training 
objectives.  Below  are  the  recommendations  for  these  purposes. 

Enhancing  the  Awareness  and  Atypical  Operational  Skills  of  Pilots  about 

HMD 

•  Inject  Events  of  Partial  HMD  or  HMD  Symbology  Failures  Requiring  a 
Crosscheck  With  Head  Down  Instruments  To  Resolve. 

•  Inject  events  of  Total  HMD  Failure  and  Train  Pilots  to  Accomplish  All 
Mission  Types  Allowed  in  SOP. 

Objective  2.3  The  pilots  should  acquire  the  capability  to  transition  safely 
back  and  forth  between  Head-Up  and  Head-Down  operations. 

Other  than  Head-Up  to  Head-Down  transitions,  the  DAS  transitions  also 
are  predicted  to  be  critical  phases  of  the  flight.  The  pilots  will  “see  through”  the 
fuselage  and  look  for  targets  of  interest  under  the  aircraft,  etc.,  and  then  they  will 
have  to  transition  back  to  normal  vision  or  vice  versa.  Problems  such  as 
disorientation  or  attentional  shift  concerns  are  predicted,  while  these  transitions 
happen  during  high  workload,  adverse  weather,  and  many  other  demanding 
situations.  The  pilots  should  be  able  to  use,  and  transition  to  or  from  the  DAS 
safely.  Another  aspect  of  this  issue  is  well-defined  operational  procedures  that 
will  enable  the  pilots  to  comprehend  the  current  situation,  and  make  proper 
assessments  depending  on  the  operational  procedures. 

Preparing  the  Pilots  for  DAS  Transitions  and  Use 

•  Present  demo  situations  concerning  use  or  non-use  of  DAS,  and  observe 
that  the  pilots  make  proper  decisions  of  when  to  use  or  not  use  the 
system. 
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•  Inject  high  workload  scenarios  requiring  the  use  of  DAS,  and  observe  that 

the  pilots  totally  follow  the  SOP  and  checklist  items  while  using  HD  and 

DAS. 

•  Train  pilots  for  adverse  weather  and  many  other  demanding  situations  of 

DAS  use.. 

•  Train  pilots  for  recoveries  after  the  problems  caused  by  improper  DAS 

use,  such  as  disorientation  and  attentional  shift/focus  problems. 

3.  Display  Suite  Management 

Objective  3. 1  The  pilots  should  acquire  full  knowledge  about  the  display 
suite.  The  raw  technical  information,  modes,  and  menus  of  the  portals,  meanings 
of  symbology,  and  the  required  skills  for  operational  procedures  are  among  the 
important  aspects  related  to  the  display  suite  management. 

Objective  3.2  The  pilots  should  be  able  to  setup,  operate  and  modify  the 
portals  effectively  according  to  the  mission  requirements. 

Objective  3.3  The  pilots  should  cross  check  the  data  shown  with  various 
sensors  if  possible,  and  be  able  to  anticipate  any  problems  about  the  symbology 
and  presented  information. 

Objective  3.4  The  pilots  should  be  ready  to  accomplish  their  tasks 
successfully  during  display  or  portal  failure. 

Objective  3.5  The  pilots  should  not  fixate  on  the  display  even  under  high 
workload  tactical  situations  and  should  also  be  aware  of  other  parameters. 

Objective  3.6  The  pilots  should  build  comprehensive  knowledge  about  the 
interactions  and  cooperation  among  the  systems  feeding  the  display  suite.  In 
case  of  a  failure  of  any  of  those  systems,  the  pilots  should  be  aware  of  the 
reliability  of  the  information  at  hand  and  make  correct  decisions  in  these 
contingency  situations. 
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The  aim  of  the  training  objectives  of  this  section  is  the  technological 
knowledge  and  operational  capabilities  related  to  the  display  suite.  How  to  use  it 
in  demanding  tactical  situations  and  the  decision-making  processes  are 
addressed  in  the  following  chapter.  There  are  two  driving  facts  that  make  the 
display  suite  a  crucial  system  in  JSF.  First,  it  will  be  the  main  interface  while 
operating  JSF  systems.  Almost  all  required  tactical  data  are  fused  and  presented 
in  this  display.  And  the  other  point  is  about  a  safety  concern  directly  related  to  the 
aforementioned  fact.  The  participants  foresee  a  potential  threat  of  fixation  on  the 
display  suite  during  operations. 

Display  suite  management  is  also  one  of  the  concerns  of  the  design  team. 
Adams  (2007)  made  interviews  with  the  design  officials  in  the  JSF  project  and 
they  confirmed  that  the  display  suite  as  one  large  LCD  screen  instead  of  the 
older  display  systems  is  one  of  the  important  innovations  in  the  JSF  cockpit.  The 
need  of  addressing  how  to  manage  this  display  suite  is  also  one  of  the  training 
objectives  of  the  design  team,  and  a  PC  Trainer  system  is  developed  for  this 
purpose. 

If  pilots  can  increase  their  proficiency  and  speed  in  managing  the  display 
suite,  it  will  free  their  valuable  cognitive  resources  during  high  workload 
situations.  They  will  be  able  to  focus  on  the  decision-making  processes  and  how 
to  acquire  the  important  data  quickly,  but  not  how  to  operate  the  display 
management  system.  Below  are  the  recommendations  for  the  objectives  related 
to  display  suite  management. 

•  Train  the  pilots  in  display  suite  management  until  they  become  proficient, 
even  under  high  workload  conditions  while  operating  the  display  suite. 

•  Make  sure  that  the  pilots  totally  understand  the  menus  and  the  symbology 
of  the  portals. 

•  Train  to  setup  the  display  suite  properly  depending  on  the  dynamic 
mission  requirements,  and  ask  for  mission  critical  data  in  high 
informational  overload  and  high  workload  conditions. 
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•  Train  the  pilots  to  crosscheck  the  presented  data  from  redundant  sensors. 

•  Train  pilots  for  display  malfunctions  and  make  sure  that  they  are  capable 

of  accomplishing  the  allowed  mission  types  under  these  abnormal 

conditions. 

4.  Tactical  Experience  and  Decision  Making 

Objective  4.1  The  pilots  should  be  proficient  enough  to  filter  the 
unnecessary  data  easily,  and  obtain  the  required  information  from  the  tactical 
displays. 

Objective  4.2  The  pilots  should  be  able  to  process  and  evaluate  the 
presented  data  quickly  and  properly. 

Objective  4.3.  The  pilots  should  have  proper  prioritization  and  attention 
allocation  skills,  and  techniques  for  highly  demanding  and  high  workload 
situations. 

Objective  4.4  The  modern  sensors  will  overload  the  pilots  with  too  much 
information,  and  it  is  likely  that  the  pilots  will  experience  many  situations  that  are 
complex  and  hard  to  analyze,  and  for  which  it  will  be  hard  to  make  proper 
decisions.  The  pilots  should  have  the  capability  to  make  quick  and  proper 
decisions  under  such  complex  scenarios. 

Objective  4.1  is  mainly  covered  in  the  previous  section,  but  it  is  also 
related  to  this  section  in  terms  of  experience.  It  also  takes  experience  to  know 
where  to  look  in  order  to  find  data.  Expert  pilots  know  where  to  look  and  when, 
and  easily  access  required  information  with  less  effort.  The  prioritization 
capabilities,  filtering  and  evaluating  the  data,  attentional  allocation,  and  decision 
making  under  high  informational  and  stress  load  are  all  interrelated.  This  is  the 
reason  to  keep  and  analyze  all  of  the  objectives  together  to  propose  solutions. 

The  important  finding  of  the  FAA  research,  as  perceived  by  these  authors, 
about  flight  training  that  is  reviewed  in  Chapter  III  is  that  shifting  the  curricula  to  a 
more  SBT  orientation,  with  real  consequences  in  order  to  address  the  complex 
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decision-making  capabilities,  has  the  potential  of  providing  the  trainees  enhanced 
cognitive  capabilities  they  will  require  during  their  demanding  tasks  in  their 
modern  cockpits  (Fiduccia  et  al.,  2003;  French  et  al. ,  2005;  Robertson  et  al., 
2006;  Dornan  et  al.,  2007).  Considering  the  finding  that  the  participants  strongly 
agree  on  the  importance  of  the  tactical  experience  in  JSF,  the  decision-making 
mechanism  of  the  experts  also  emerges  as  crucial  for  JSF  pilots. 

The  recommendations  about  the  tactical  experience  and  decision-making 
in  JSF  are  as  follows: 

Recommendations  for  Decision  Making  and  Acquiring  Tactical 
Experience  Faster 

•  Present  pilots  with  tactical  pictures  on  the  ground  and  ask  for  specific 
decisions.  This  can  be  accomplished  both  by  using  static  pictures  of  the 
displays  or  simulator  and  desktop  trainers. 

•  Various  demanding  tactical  pictures  along  with  critical  decisions  can  be 
discussed  in  classrooms. 

•  Inject  high  workload  into  the  mission  scenarios  and  train  pilots  on  fixation 
issues. 

•  Train  pilots  for  proper  prioritization  and  attentional  allocation  skills 
depending  on  the  mission  demands,  task  at  hand,  and  workload.  Inject 
various  demanding  tactical  scenarios  to  observe  proper  skills  by  the 
trainees. 

•  The  terminology  used  during  the  mission  communications,  how  to  operate 
the  displays,  filter,  acquire  the  data,  and  making  decisions  are  considered 
to  be  taught  via  simulators  and  desktop  trainers  with  HOTAS;  thus, 
missions  such  as  BVR  engagements,  SEAD,  and  many  more  can  be 
taught  on  the  ground.  The  pilots  will  struggle  less  with  the  terminology, 
display  management,  and  decision-making  in  the  air  after  such  training 
with  effective  debriefings.  And  their  tactical  experience  will  build  up  earlier. 
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D.  CONCLUSIONS 

Throughout  the  thesis  research,  the  authors  always  aimed  to  both  identify 
and  solve  the  potential  human  factors  issues  in  JSF.  Although  the  initial  thesis 
plan  was  built  around  the  potential  NTT  issues  during  transition  periods  only,  the 
initial  and  follow-on  surveys  did  not  predict  it  as  an  issue. 

The  potential  concerns  identified  by  this  thesis  are:  the  possible  problems 
during  the  use  of  automation,  potential  disorientation  concerns  in  DAS  use, 
display  suite  management,  and  complex  cognitive  skills  required  both  for  tactical 
decision-making  and  information  management  processes. 

The  research  literature  review,  subjective  reports  by  the  participants  and 
the  operational  experience  of  these  authors  formed  the  basis  for  the 
recommendations  on  pilot  training  in  JSF.  A  curriculum  revision  from  the 
traditional  training  curriculums  is  recommended  to  address  both  the  potential 
automation-related  problems  and  to  ensure  that  the  pilots  build  relevant  cognitive 
skills,  by  utilizing  the  ground  training  devices  as  much  as  possible. 

E.  FUTURE  RESEARCH  RECOMMENDATIONS 

One  of  the  two  objectives  of  this  thesis  is  to  identify  the  human  factors 
concerns  in  JSF,  and  the  other  is  to  develop  recommendations  for  training  to 
solve  the  concerns.  In  the  follow-on  survey  analysis  chapter,  the  shortcomings  of 
the  survey  were  provided,  but  there  remain  many  other  methods  for  problem 
identification  and  solution  creations.  The  aim  of  this  chapter  is  to  provide  the 
framework  for  possible  future  research  for  both  processes. 

1.  Problem  Identification 

The  most  important  phase  of  problem  solution  is  to  understand  the 
situation  first.  If  this  step  is  not  totally  fulfilled,  one  cannot  propose  robust 
solutions.  This  thesis  only  had  the  opportunity  to  benefit  from  limited  resources 
for  a  subjective  method  for  problem  identification,  and  what  could  be  done  in 
order  to  have  a  better  subjective  research  is  briefly  discussed  in  Chapter  V. 
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a.  Research  Recommendations 

•  In  all  probability,  there  are  ongoing  analyses  and  surveys  at 
Lockheed  Martin  about  usability,  workload,  and  many  other 
concerns,  as  well  as  their  solutions  related  to  human  factors 
concerns  in  JSF.  The  decision-making  skills  of  the  test  pilots  may 
be  captured  in  highly  demanding  complex  situations,  and  other  than 
establishing  SOP,  normal,  and  emergency  operational  procedures. 
These  skills  may  be  imported  to  JSF  training.  This  thesis  both 
supports  the  ongoing  studies  and  provides  new  insights  on  them. 

•  JSF  will  be  flown  in  many  allied  countries.  Each  country  may  have 
differences  in  terms  of  pilot  training,  and  subjective  studies  may  be 
conducted  in  order  to  identify  various  concerns  depending  on  the 
countries  or  types  of  aircraft.  This  research  will  yield  different 
results  depending  on  the  background;  thus,  the  training  curriculums 
can  be  tailored  to  specific  needs  if  necessary. 

•  The  pilots  with  experience  with  the  JSF  simulator,  desktop  trainer 
or  actual  aircraft  may  be  also  surveyed  for  their  training 
recommendations  to  identified  problems. 

2.  Objective  Methods 

Another  methodology  is  the  objective  research.  Many  experimental 
designs  could  serve  this  objective,  but  the  limitations  prevented  these  authors 
from  getting  access  to  any  JSF  trainer  or  simulator  in  order  to  conduct  an 
experiment.  The  design  team,  and  many  other  frontier  officials  already  are 
involved  with  this  objective,  but  because  of  the  security  issues,  the  authors  do 
not  have  robust  knowledge  what  has  been  done.  But  one  important  fact  is  that 
only  the  design  team,  JSF  program  officials,  and  those  researchers  having 
access  can  do  these  experimental  studies. 
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The  authors  do  not  have  enough  knowledge  about  the  capabilities  of  the 
actual  aircraft,  full  mission  simulator,  part  task  trainers,  or  desktop  trainers  in 
terms  of  conducting  an  experiment;  thus,  it  would  not  be  legitimate  to  propose  an 
experimental  design.  The  recommendations  below  are  only  suggested  as  the 
framework  for  possible  experiments  for  further  research. 

a.  Recommendations  for  Experimental  Research 

•  All  of  the  findings  of  this  thesis,  or  any  other  research  related  to 
JSF  pilot  concerns,  can  be  evaluated  using  various  methods,  such 
as  simulation. 

•  For  automation  findings,  scenarios  with  various  mission 

requirements,  informational  overload,  and  workload  level  can  be 
injected  into  controlled  scenarios  and  while  accomplishing  the 
tasks;  the  awareness  levels  of  pilots  with  automated  variables  can 
be  measured.  Also,  demanding  atypical  procedures  can  be  injected 
into  experimental  scenarios  to  measure  the  participants’  complex 
decision  making  skills  about  automation;  thus,  it  can  be  determined 
whether  their  mental  models  are  good  enough.  The  second  step 
after  these  experiments  would  be  testing  the  solutions.  After 
establishing  proper  operational  procedures  to  address  the 
problems,  and  providing  the  appropriate  training,  similar 
experiments  would  serve  also  for  testing  whether  the  training  is 
effective  or  not.  A  control  group  with  no  training,  and  other  groups 
with  other  types  of  training  will  be  beneficial  to  understanding 
further  the  effectiveness  of  the  proposed  training  solutions. 

•  For  DAS  use,  the  experiments  depend  on  the  simulator  capabilities. 
The  required  extent  for  motion,  and  visual  cues  to  create  situations 
to  the  real  aircraft  are  crucial  in  order  to  conduct  an  experiment  on 
the  ground.  That  has  to  be  validated  by  the  test  pilots,  or  any  other 
source  as  possible.  If  the  simulators  are  good  enough,  various 
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scenarios  in  day  and  night  conditions,  and  in  adverse  weather 
conditions  with  dynamic  maneuvers  should  be  constructed  where 
the  DAS  use  is  either  a  choice  or  mandatory.  If  the  situation 
requires  a  choice,  whether  the  pilots  face  difficulties  as  predicted  or 
not  can  be  tested.  And  if  the  use  is  mandatory  by  mission 
requirements,  the  situations  prone  to  mishaps  can  be  also  identified 
objectively.  The  following  step  would  be  to  test  the  training 
solutions.  Similar  scenarios  can  be  constructed  for  trained  pilots  in 
order  to  measure  their  performance,  making  sure  that  the  issues 
are  addressed  effectively. 

•  For  display  management  and  tactical  decision  making  capabilities, 
two  important  questions  are  whether  the  pilots  could  build 
experience  with  ground  training  devices  effectively  or  not,  and 
whether  proposed  newer  learning  curriculums  will  help  pilots  with 
their  complex  decision  making  skills  or  not.  For  the  second 
question,  a  group  with  similar  training  as  F-16,  F-15,  or  many  other 
current  type-training  curricula  can  be  used,  and  the  JSF  group  will 
be  trained  with  the  proposed  curricula.  The  differences  can  be 
tested  both  with  decision-making  scenarios,  where  the  pilots  are 
asked  to  make  decisions  to  given  problems.  Informational  overload 
also  can  be  evaluated. 

•  Other  than  the  decision  making  process,  the  effect  of  training 
devices  on  tactical  experience  is  also  an  important  question.  The 
main  idea  is  that  the  experts  make  quick  and  proper  decisions  with 
less  effort  from  their  experiences.  The  end-result  of  this 
phenomenon  is  that  if  a  pilot  makes  a  decision  fast  and  properly, 
and  understands  the  current  tactical  situation  easily,  it  can  be 
claimed  that  he  has  enough  tactical  capability.  Two  groups  could  be 
used  to  measure  this  question,  one  group  with  proposed 
curriculum,  using  training  devices  for  tactical  scenarios  as 
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explained  in  training  recommendations  section,  and  the  second 
group  with  a  traditional  training  curriculum.  Both  groups’ 
performance  can  be  measured  during  actual  tactical  missions,  and 
either  the  elapsed  time  until  a  certain  goal  can  be  measured,  or  at  a 
certain  time  the  performance  differences  can  be  measured. 

It  is  certain  that  many  of  these  recommendations  have  been  considered 
and  many  others  are  implemented  by  the  officials  in  the  JSF  project,  but  research 
from  various  sources  is  helpful  to  increase  confidence,  or  to  gain  more  insight 
into  the  situations.  Also,  many  important  points  are  expected  to  emerge  after  the 
aircraft  begins  to  fly  in  the  air  forces,  but  the  goal  for  researchers  and  related 
officials  should  be  to  predict  actual  problems  in  advance,  and  solve  them  prior  to 
use. 
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